ABSTRACT 



DIFFUSION OF INNOVATION: A CASE STUDY OF 
COURSE MANAGEMENT SYSTEM ADOPTION 



Phillip Powell, Ed.D. 

Department of Educational Technology, Research and Assessment 
Northern Illinois University, 2008 
Lara Luetkehans, Director 

The purpose of this study was to examine the processes of evaluation, 
selection, adoption and diffusion of a course management system (CMS). This study 
incorporated two institutions, each having six participants. The participants were 
placed into three categories: faculty, staff, and administration. Each participant was 
either a faculty member, who was also an early adopter of the institution’s CMS, a 
staff member, or an administrator. Each participant was involved in part or all of the 
evaluation, selection, adoption and diffusion processes. 

The outcome of the study indicated that the participation of these individuals 
helped the institutions evaluate and select their CMS. In addition, the study indicated 
that each institution arrived at their decisions through different evaluation and 



selection processes. University #1 consisted of a committee to evaluate and select their 




system, while University #2 comprised members of the Technology Department and 
utilized the input of faculty members to help select a system. The findings also 
recommended that the participation of these individuals influenced others to adopt the 
system. Each university, however, provided different incentives to entice the adoption 
of these systems. 

The study recommends that institutions with the resources might be more 
suitable to adopt open-source systems than smaller institutions without adequate 
resources to maintain these systems. Furthermore, the research suggests that additional 
studies are needed to analyze how many small institutions are using open-source 
systems and to what extent are they using these systems. 
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PREFACE 



Researcher’s Background 

My introduction to course management systems (CMS) began in the summer 
of 2000 as a student in an instructional technology course. The objective was to 
develop a prototype of an on-line course, utilizing a selected CMS or web-based 
authoring tool. I chose Blackboard, one of many CMSs, because it took very little time 
to learn and featured a user-friendly interface. As a faculty member in the 
Communication Department, I chose to develop an on-line version of a broadcast 
programming course I taught in the fall semester. As I uploaded course materials 
(syllabus, documents, and other files) onto Blackboard, I realized I would need to 
upload video samples, so I sought the advice of various technology administrators 
concerning packaging these materials for placement onto the Blackboard site. 
Unfortunately, I found they were aware of the CMS concept, but the university had 
not subscribed to any proprietary vendor. 

Ironically, the institution was considering adopting a CMS and encouraged 
faculty to participate in the process. With my coincidental introduction to CMS’s and 
the timing of the institution’s decision to adopt a CMS, I became interested in 
developing a dissertation topic that involved the process of evaluation, selection. 




adoption, and diffusion of a CMS and examined the experiences of stakeholders who 
participated in the process of analysis, selection, adoption, and diffusion of a CMS. 

Since my introduction to the Internet in 1994, 1 have become increasingly 
aware of its impact on learning. I strongly believe distance education has evolved 
beyond classroom settings, instead relying heavily on CMS and Internet-based 
applications. The use of CMS continues to grow and as a researcher I have a vested 
interest in following its growth and continuing to research its impact on learning 
environments. 

As a researcher, I became consciously aware of University #l’s and University 
#2’s decisions to select a CMS. However, my awareness of their decision and the 
understanding of the process did not influence their participation in this study. With 
my knowledge and understanding of the CMS evaluation and selection process, I have 
a biased opinion, yet my opinion did not influence the outcome of this study. 




CHAPTER 1 



INTRODUCTION 

In the last decade, the field of distance education has grown and changed 
dynamically (Gunawardena & Mclsaac, 2004, p. 369). During the early 1990s, many 
institutions of higher education re-tooled their distance education courses by offering 
access to instructional materials via the Internet. Technological advancements also 
changed interaction between the instructor and learner, altering the communication 
process. 

Over the last thirty years, distance education has grown to reach a wide range 
of clientele (Bates & Poole, 2003, p. 126). For example, computer-mediated 
communication (CMC) offers around-the-clock access to instructional materials via 
the web, and provides access to course content including the development of 
pedagogical web-based applications such as e-learning (electronic learning) or web- 
enhanced learning, characterized by “the creation of education and learning 
arrangements where the Internet plays an important role in the delivery, support, 
administration, and assessment of learning” (Kirschner & Pass, 2001, p. 350). 

The success of e-learning depends upon the use of technology such as laptop 
computers, wireless networks, and wireless tools (phones, PDAs, etc.), and as these 
tools become more accessible due to decreasing costs and widespread availabilities, it 
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will ultimately enhance a learner’s abilities to effectively organize learning content. 
These technologies associated with e-learning enable new pedagogy and provide 
alternative ways to teach and learn. 

With the growth of tools to enhance e-learning pedagogy, more college 
campuses have integrated computer-based technology into courses. For example, 
software technology tools such as course management systems (CMS) are utilized by 
instructors to enhance e-leaming (Waterhouse, 2005, p.5). CMSs help to facilitate e- 
leaming by incorporating instructional features such as course-related announcements, 
course syllabi, course outlines, projects, messages, asynchronous/synchronous chat 
capabilities, and access to e-mail accounts while handling various administrative 
course management functions. 

The development and use of CMS began in the early 1960s. At that time, 
computerized learning systems were developed as a means to provide access to 
interactive learning modules. One early system was PLATO (Program Logic for 
Automated Teaching Operations), originally built by scientists at the University of 
Illinois. PLATO helped to launch tools for online interaction, including a means of 
posting messages and creating forums, communicating through email and chat rooms, 
creating interaction through remote screen sharing, and playing online games 
( http://en.wikipedia.org/wiki/PLATQ) . Many years later, the concept of online 
education grew and became enriched through organizations such as CALCampus, the 
Computer Assisted Learning Center (CALC). Founded in 1982 in Rindge, New 



Hampshire, as a small, offline computer-based adult learning center, the CALCampus 
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philosophy was similar to the model of today’s online education, in that CALCampus 
sought “to provide affordable, quality instruction to individual learners through the use 
of computers” (Morabito, 2008, p.l). Organizations and companies such as 
CALCampus helped launch today’s CMSs, which include tools such as Blackboard 
Learning Systems, Moodle, Angel Learning Management Systems, eCollege, and 
Sakai. In addition, virtual universities were created to attract students who may not be 
able to attend traditional colleges and universities due to various reasons. A list of 
institutions includes California Virtual Campus, Ohio Learning Network, Michigan 
Virtual University, Minnesota Virtual University, Phoenix University Online, Walden 
University, Western Governors University, and University of Maryland University 
College. Acceptance and use of course management tools is illustrated in Figure 1. 
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Figure 1. Course Management Tools 2000-2003 (Green, 2003). 
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The blend of audiences utilizing CMS includes faculty, students, staff, and 
administration. Educational institutions recognize this as a very important tool for 
distance education delivery, but each institution is different in that some may have 
standardized systems while others allow the use of systems chosen or developed by 
individual users or departments. In addition, the value of course management systems 
has increased in recent years partly due to the ability to reach a wider audience. 
According to a 2003 survey conducted by the Campus Computing Project: 



... 33.6 percent of all college courses now use course management 
tools, up from 26.5 percent in 2002, 20.6 percent in 2001, and almost 
double the level in 2000 (14.7 percent). The survey data also reveal that 
over half (51.4 percent) of the survey respondents report that their 
institution has a strategic plan for deploying course management tools, 
compared to 47.5 percent in 2002. More than four-fifths (82.3 percent) 
of the participating institutions have already established a “single 
product” standard for course management software, up from roughly 
three-fourths (73.2 percent) in the 2001 survey and 57.8 percent in 
2000. (Green, 2003, p. 4) 



Statement of the Problem 



Institutions evaluate, select, and adopt a CMS, but little research has been done 
to clarify the processes and experiences of participants involved. Most research 
emphasizes CMS adoption (Morgan, 2003; West, Waddoups & Graham, 2006) and 
present models to explain technology adoption (Burkman, 1987; Hall & Hord, 1987; 
Rogers, 2003). 

Institutions are also concerned with the rising costs of maintaining systems that 
provide online course management only. Furthermore, many institutions still use 
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separate systems that handle institutional functions (i.e., financial, registration, etc.). 
The cost of maintaining both has many institutions considering a single integrated 
system. 

Most CMS vendors address the integration issue by providing a system that 
has a feature called documented application program interface (API) that allows the 
integration of CMS with the institutional student information system (SIS). 
Unfortunately, it provides only the ability to integrate, not the actual program to 
perform the integration, which means either designating the institution’s programmers 
to perform the task or purchasing professional services (Shelton & Saltsman, 2005, 
p.l28). Accordingly, choosing a CMS is a difficult decision-making process for 
technology administrators (Jafari, 2000, p.55). And furthermore, until vendors begin to 
offer services similar to campus portals, institutions face the challenge of analyzing 
and selecting the right system to fit their needs, or choosing to develop their own. 

Purpose of the Study 

This study examined the process undertaken by two institutions that selected a 
course management system and provides insight into participant experiences during 
the selection process. This study includes additional literature that supports the process 
and experiences of participants involved in selecting CMS. In addition, this study 
provides an in-depth analysis of the processes and the experiences of participants 
during the evaluation, selection, adoption and diffusion of a CMS. 
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Research Questions 

The diffusion of innovation theory was used to explain the processes of 
evaluation, selection, adoption and diffusion of a CMS. The following research 
question was the focus of this study: 

What are the factors that guide the decisions in selection and use of a CMS 
within two small liberal arts institutions? 

The following subquestions were used to guide this study. 

1 . What were the issues leading to the need for a CMS? 

2. What kinds of procedures were used in the evaluation process? 

3. What were the roles of the administration, staff, and faculty in the 
evaluation and selection processes? 

4. How was the CMS selected by the organization? 

5. How is the CMS being used in the organization? 

Significance of the Study 

This study provides an in-depth analysis of the processes and the experiences 
of participants during the evaluation, selection, adoption and diffusion of a CMS. 
Moreover, this research attempts to raise the consciousness of key decision makers to 
become more aware of issues related to these processes. With institutions considering 
the adoption of homegrown or open-source CMS, decision makers will also find this 
study useful for planning the selection and implementation of these systems. In 
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addition, with institutions considering the adoption of technological innovations, 
particularly CMSs, this study adds to the literature concerned with the evaluation, 
selection, adoption and diffusion of these innovations, as well as literature related to 
the concept of technological change within institutions. 

Definition of Terms 

For the purpose of this study, the following terms were used: 

Adoption is seen as the decision to make full use of an innovation (Rogers, 
1995, p. 21). 

Course management system (CMS) refers to web-authoring tools that integrate 
technological and pedagogical features of the Internet and the World Wide Web into a 
single, template-based authoring system to facilitate the design, development, 
delivery, and management of web-based courses and on-line learning environments 
(Dabbagh, 2002, 'll 5). This concept is also referred to as a content management 
system. 

Computer-mediated communication (CMC) is any form of communication 
between two or more individuals who interact and/or influence each other via 
computer-supported media (i.e., instant messaging, e-mail, chat rooms, etc.). 

Diffusion is defined as the process by which an innovation is communicated 
through certain channels over time among the members of a social system (Rogers, 



1995, p.5). 
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Electronic learning (e-learning) refers to using instructional strategies 
enhanced by technology (Waterhouse, 2005, p.3). 

Innovation is defined as an idea, practice or object that is perceived as new, 
whether or not it is objectively new as measured by the lapse of time since its first use 
or discovery (Rogers, 1995, p.l 1). 

Organization of Chapters 

This dissertation is organized into five chapters. Chapter One is an introduction 
to this study, which includes the background, the statement of problem, purpose of this 
study, the significance of this the study, and definition of terms. Chapter Two 
summarizes the literature related to this study including a discussion of the diffusion 
of innovation theory and other related models. Chapter Three contains a description of 
the methodology used in gathering and analyzing data. This chapter also includes a 
description of the organization of data. Chapter Four describes the findings within this 
study and concludes with a cross-case comparison of findings. Chapter Five 
summarizes a discussion of the theoretical framework used in the design of the study, 
and offers recommendations and suggestions for future research. 




CHAPTER 2 



REVIEW OE LITERATURE 
Introduction 

The purpose of this literature review is to propose a conceptual framework to 
help identify factors related to the process of evaluating, selecting, adopting and 
diffusing a CMS. This chapter is divided into several literature areas including a 
discussion of course management systems, the diffusion of innovation theory, and 
examples of models related to change, adoption, and diffusion within a decision- 
making process. 



Course Management Systems 

Course management systems are software systems used by students to interact 
on-line through synchronous and asynchronous communication methods, by accessing 
course materials, and communicating with others (Dugas, 2005, p.l 1). In the mid- 
1990s, several university and commercial CMSs were launched either to be used fully 
online or as supplemental systems for traditional face-to-face instruction (Harrington, 



Gordan & Schibik, 2004, 'll 4). 
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The popularity, as Britto (2005) proposes, is seen as a simplistic manner in 
which content is created and delivered online and provides the opportunity for 
students to communicate either asynchronously or real-time, without needing 
knowledge of programming and web development skills ( p.70). 

In addition, Sunil (1998) suggests that CMS’s can be used to put text, graphics, 
video, and audio files onto a web site ('ll 7). 

Pedagogy and Design of CMS 

Many studies about CMS highlight the importance of pedagogy, design, and 
use. For example, Dabbagh and Schmitt (1998) examined the pedagogical re- 
engineering of a course designed for traditional learning and concluded that the 
technological components of the web-based authoring tool enabled instruction that 
was not previously perceived as possible in a traditional mode (p. 109). McGee, Suter, 
and Gurrie (2005) agree that the system determines pedagogy and interaction, and is 
tool driven (p. 155). In contrast, Ullman and Rabinowitz (2004) argue that university 
teachers approach using a CMS with a “pre-existing mental model of how to use 
technology and using it to reinforce traditional teaching styles” (p. 2). 

Other studies indicate the design of on-line CMS’s may have an effect on 
learning. Yip (2004) and Basile and D’ Aquila (2002) point out that students are 
comfortable in courses that use CMS. But in contrast, Oliver (2001) believes that 
while a CMS helps faculty to distribute information, it does not take advantage of the 
Internet’s ability to provide active learning (p. 49). Koszalka and Ganesan (2004) add 





11 



that course developers are provided with tools that help facilitate information sharing 
and eommunieation amongst users; but that does not guarantee that the learner will 
meet the learning expectations (p. 245). In essence, these studies indicate that students 
may feel eomfortable using a CMS but it may not have an effect on their learning 
capabilities. 



Faculty Use of CMS 



A few studies have been eonducted to analyze the use of a CMS by faeulty. 
Morgan (2003) addressed this by surveying 730 faeulty members and instruetional 
staff within the University of Wisconsin educational system, focusing particularly on 
adoption. Morgan (2003) concluded that most faeulty adopt a CMS mainly for users to 
access content on-line (course syllabi, announcements, etc.), in hopes to help decrease 
the time needed to provide feedback, and to facilitate more eommunieation between 
instructor and student (p.75). In 2005, a similar study was eonducted by the University 
System of Georgia to assess how institutions in the system were using instructional 
technologies, particularly CMS. Research questions originally used in the Morgan 
(2003) study were slightly altered to reveal information direetly related to faculty use 
of instructional technologies within the University System of Georgia. Results of this 
study indieated that nearly half of USG faculty were using a CMS and nearly two- 
thirds inereased their use of a CMS over time. In addition, two-thirds of the users 
indicated using a CMS increased the engagement of students in their learning 
(Gastfriend & Finnegan, 2005). In 2006, The Campus Computing Project conducted a 




12 



national survey of information technologies in United States higher education. Results 
revealed a small percentage of private and public institutions were using open-source 
CMS. The study indicated 10.2% of private institutions had adopted Moodle, while 
5.5% of public institutions and 3.9% of private institutions were using Sakai. Green 
(2006) adds that while open-source systems are in their infancy, more institutions 
would become adopters as information technologists share their experiences with 
others (p.3). This trend also indicates more research is needed to address concerns 
about what type of information is included in an institution’s decisions to adopt these 
systems, and how these institutions arrived at their decisions to adopt an open-source 
system. 

Additional studies related to CMS use also reveal how these systems are being 
used. For instance, Ansorge and Bendus (2003) and Woods, Baker, and Hopper (2004) 
all agree that faculty use CMS to transmit documents to students. However, additional 
research also indicates adoption and use may be affected by faculty concerns. For 
example, McQuiggan (2006) employed the Stages of Concerns (SoCO) Questionnaire 
(Hall, George, & Rutherford, 1998) to describe university faculty innovation concerns 
and perceptions that influence the adoption of a CMS. The research focused on 
distinguishing the adopters and non-adopters perceptions of using a CMS, based on 
four sets of constructs: individual characteristics, individual concerns about the CMS, 
individual perceptions about the CMS attributes; and individual communication 
channels (McQuiggan, 2006, p. 1 164). The study found a significant difference 
between the adopters and non- adopters in that the adopters’ perceptions indicated 
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using the CMS reflected a higher level of advantage compared to previous 
pedagogical practices. The study also indicated that adopters identified using a CMS 
as more compatible with their teaching styles, felt a sense of ease to learn and use the 
CMS, and exhibited a willingness to see it demonstrated and try it before using it. In 
contrast, the non-adopters did not use technology and felt using a CMS was 
incompatible with their face-to-face interaction with students (McQuiggan, 2006, 
p.1165). 

Although these studies suggest pedagogical design and motivation may 
influence use, and that faculty may exhibit concerns when adopting a CMS, 
unfortunately there is still a void in literature to help explain the processes of 
evaluation, selection, adoption and diffusion. There is, however, a growing amount of 
literature that helps explain the adoption and diffusion of technology, including the 
diffusion of innovation theory (Rogers, 1995, 2003) as a theoretical framework for 
many research studies. 



Diffusion Theory 

A proliferation of literature has been generated to explain the adoption and 
diffusion of technology including Zaltman and Duncan’s (1977) resistance/adoption 
model; Hall and Hord (1987) concerns-based adoption model (CBAM); and 
Burkman’s (1987) user-oriented instructional development process. However, Roger’s 
(1995, 2003) diffusion of innovation theory is most relevant to describing the 
evaluation, selection, adoption, and diffusion process, using a combined explanation 
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of theories including the innovation decision process and perceived attributes of an 
innovation. 



Diffusion Research 

Diffusion research can be found in various fields of study: education, 
sociology, public health, communication, anthropology, industrial engineering, 
geography, general economics, public administration and political science (see 
Rogers, 1995, p.42). Rogers sees this as a convergence of intellect, which comprises a 
cross-disciplinary approach (p.39). Everett M. Rogers has been credited as the author 
whose studies have led to many investigations of diffusion. Rogers credits his 
inquiries to Ryan and Gross’s (1943, 1950) research on the diffusion of hybrid com, 
featuring a qualitative analysis of the amount of time it took Iowa farmers to adopt the 
innovation of hybrid corn seeds. The outcome evolved into a research paradigm and 
research methodology that represents the present academic template of Diffusion of 
Innovation theory (Rogers, 1995, p.54). 

Explanation of the Diffusion of Innovation Theory 

Diffusion of innovation theory has been applied in many areas such as 
instructional technology, electronic publishing, and media literacy programs and has 
assisted in developing models and theories to explain the process of adoption and 



diffusion. 
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The process of diffusion involves communication between change agents to 
encourage the adoption of an innovation by an individual, organization, or other 
adoption unit through social systems. Diffusion also includes the awareness and 
knowledge of an innovation, and focuses on examining attitude change, decision 
making, and implementation of the innovation (Sutherland, 2003, 'll 6). The real 
challenge is to identify and implement the correct change (Oliver, 1996, p.7). 

The concept considers three main components: the innovation, the diffusion 
process, and adoption. The innovation is defined as “an idea, practice or object that is 
perceived as new, whether or not it is objectively new as measured by the lapse of 
time since its first use or discovery (Rogers, 2003, p.l2). Diffusion is defined as “the 
process by which an innovation is communicated through certain channels over time 
among the members of a social system” (p.5). Adoption is seen as “the decision to 
make full use of an innovation as the best course of action available” (p. 21). 

Rogers (1995) adds that newer alternatives motivate users to further investigate 
information about an innovation because of the uncertainty created (p.xvii). This 
process allows the flow of communication to continue until each party involved adopts 
or rejects the innovation. Rogers describes this form of acceptance or rejection as 
homophily or heterophily. Homophily occurs because individuals share the same 
interests or belong to the same groups; heterophily occurs when two or more 
individuals are opposite in nature. Thus, when similar individuals identify with the 
technology, acceptance occurs; however, heterophily (rejection) occurs when the 
parties are of different educational or social status and consequently do not speak the 
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same language (1995, p.l9). Respectively, either could lead to success or failure of 
adoption of an innovation. 



Adopter Categories 

Other factors of the diffusion of innovation theory include characteristics of an 
innovation and adopter categories. Rogers classifies adopters of innovations into five 
categories: innovators, early adopters, early majority, late majority, and laggards. In 
relationship to technology adoption, these categories correspond to times in which 
users identify a technology and begin to adopt the technology. The characteristics 
relates to the relative advantage of the innovation and the degree to which it is 
perceived to be better than what it supersedes; compatibility and consistency with 
existing values, past experiences and needs; complexity and difficulty of 
understanding and use; trialability, the degree to which it can be experimented with on 
a limited basis; and observability, the visibility of its results (Clarke, 1999, ^ 5). 

Adopter categories portray participants as venturesome, respectable, deliberate, 
skeptical, and traditional. Rogers (1995) gives an example of the adopter categories as 
an analogy of a bell curve which begins at one extreme with some people willing to 
try the innovation, and ending with people who reluctantly take the time to review and 
accept the innovation (Rogers, 1995, p.262). This behavioral pattern is described as 
innovativeness, a bottom-line type of behavior in the diffusion process, as illustrated 



in Figure 2 (Rogers, 1995, p.262). 
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Diffusion of Innovation Framework 

Four main elements comprise the Diffusion of Innovation framework: (1) the 
innovation and perceived attributes, (2) communication channels, (3) time, and (4) a 
social system (Rogers, 1995, p.ll). The first element, the innovation, addresses the 
significance of the innovation in terms of: (a) how early adopters differ from later 
adopters, (b) perceived attributes concerning relative advantages which affect the rate 
of adoption, and (c) an explanation of when adopters begin to use the innovation 
(Rogers, 1995, p. 12). The second element, communication channel, develops the 
means by which the message of the innovation gets from one individual to another. 
The third element, time, considers three things: (a) the innovation decision process in 
which individuals are introduced to the innovation and then choose to adopt or reject 
the innovation, (b) the early or later time in which the individual adopts an innovation, 
and (c) the number of individuals who adopt an innovation in a period of time. The 



fourth element, social system, involves a “set of interrelated units that are engaged in 
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joint problem-solving to accomplish a common goal” (Rogers, 1995, p. 23). This 
includes individuals, informal groups, organizations, and subsystems. These four 
elements of the Diffusion of Innovation framework (Figure 3) comprise what Rogers 
describes as a model of the innovation decision process. 

Innovation Decision Process 

The innovation decision process consists of five stages: (1) knowledge, (2) 
persuasion, (3) decision, (4) implementation, and (5) confirmation, and is 
characterized as a process that occurs while individuals participate in a series of 
actions related to decisions (Rogers, 1995, p. 162). Knowledge occurs when 
individuals are aware of the innovation and gain understanding of its functions. 
Persuasion is when individuals or decision-making units exhibit favorable or 
unfavorable behavior toward the innovation. Decision indicates when the individual or 
unit decides to adopt or reject the innovation. Implementation occurs when the 
individual or unit decides to use the innovation. Confirmation occurs when decision 
makers confirm or reject their decision to adopt the innovation (Rogers, 1995, p.l62). 
The characteristics of this model emphasize a way of seeking and processing 
information about an innovation and motivating individuals to possibly reduce 
anxieties towards adopting an innovation, thus creating a greater understanding. 

Rogers validated these findings based on his research on the Iowa study (Beal & 
Rogers, 1960), which found that respondents 
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Figure 3 . diffusion of innovations framework (based on Rogers, 1995). 
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indicated going through these stages, and also similar findings reported from other 
studies, e.g., Wilkening (1956), Mason (1962), and Singh and Pareek (1968). 

Rogers (1995) adds that the change of strongly held attitudes usually occurs 
through interpersonal channels and becomes more important at the persuasion stage in 
order to implement change within the innovation decision process (p.203). 

Models of Diffusion 

Other decision-making models related to diffusion include: Zaltman and 
Duncan’s (1977) resistance/adoption model. Hall and Hord (1987) concerns-based 
adoption model (CBAM), and Burkman’s (1987) user-oriented instructional 
development process. These models stress various aspects of the adoption-diffusion 
process as described by Roger’s. The following section will review each model and 
the rationale for the use of Rogers’s model is offered. 

Zaltman and Dunean Resistanee/ Adoption Model 

According to Zaltman and Duncan (1977), an innovation is the “change object 
and change is the alteration in the structure of a system that requires or could require 
relearning on the part of the actors in response to the innovation” (p. 12). Like 
Rogers’s innovative decision process, Zaltman and Duncan’s resistance/adoption 
model include eight stages of the decision-making process: perception, motivation, 
attitude, legitimation, trial, evaluation, adoption/rejection, and resolution. The process 



begins with the introduction of the innovation and perceived needs for it by the 
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adopter. Motivation occurs when the individual is overcoming resistance to the 
innovation and changing existing habits to embrace the innovation. The attitude stage 
develops three components: cognitive, affective, and behavioral beliefs of the 
innovation, including the retention and reinforcement of beliefs throughout the 
decision process. Legitimation transpires when the adopter seeks reinforcement for an 
action being considered. Trial consists of a personal test of the innovation; evaluations 
point out the pros and cons of the innovation and is usually important before the 
adoption stage. During the adoption/rejection stage, an individual normally makes a 
commitment or may even reject the innovation due to satisfactory or unsatisfactory 
outcomes. The final stage concludes with resolution, which reduces dissonance and 
evaluates the change fully (Zaltman & Duncan, 1977, p. 237). 

Comparison of Rogers’s and Zaltman and Duncan’s Models 

Zaltman and Duncan’s model reflects two stages similar to Rogers’s, as 
illustrated in Figure 4. 



Rogers’s Model 


Zaltman and Duncan’s Model 


Knowledge 


^ Perception 


Confirmation 


► Legitimation 



Figure 4 . Comparison of Rogers’s and Zaltman and Duncan’s models. 







22 



Knowledge and perception indicates the need for change has been 
communicated and an innovation has been identified for adoption, whereas 
confirmation/legitimation occurs after the innovation has been evaluated and a 
decision has been made. The conflict in these stages suggests that motivation and 
attitude were not described as complete stages like Rogers’s persuasion and decision, 
but were perceived as, feelings by the individuals on how much control they had over 
how the innovation affected them. Resolution was not applicable to Rogers’s 
framework because rather than implementation and confirmation, it seeks 
discontinuance, usually leading to rejecting an innovation after the innovation has 
been previously adopted. 

Rogers’s model allows an individual to advance through stages of the decision- 
making process based on the end result being implementation and adoption, whereas 
Zaltman and Duncan see the decision process ending with the choice of adopting an 
innovation and later rejecting it. Zaltman and Duncan’s model indicates that the 
decision to adopt can be achieved, but as Rogers demonstrates, implementation must 
occur in order for the innovation to be utilized. 

Concerns-Based Adoption Model 

The concerns-based adoption model (CBAM) originated through research 
conducted at the Research and Development Center for Teacher Education at the 
University of Texas in Austin in the early 1970s. Specifically developed with the 
concerns of teachers in mind as they adopt and implement educational innovations, the 
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CB AM model represents an explanation of how change facilitators interact with 
individuals and groups to intervene in actions that affect teachers’ use of new 
programs and practices (Hall & Hord, 1987, p.l6). Three dimensions of the CBAM 
model are used as diagnostic tools to facilitate use of the innovations, programs, and 
practices: Stages of Concerns (SoC), Levels of Use (LoU), and Innovation 
Configurations. These dimensions represent key aspects of the change process as 
experienced by individuals (Hall & Hord, 1987, p.l3). The first dimension. Stages of 
Concerns (Figure 5), involves seven different stages that range from self-type 
concerns, which are teacher-focused, to task-related concerns addressing logistics and 
scheduling in regards to the use of an innovation, to impact which represents benefits 
and consequences of use (p. 13). These stages include: (0) awareness, 

(1) informational, (2) personal, (3) management, (4) consequences, (5) collaboration, 
and (6) refocusing (p.60). The first stage has little impact because no concern or 
involvement is indicated. It then moves to informational, which begins a general 
awareness and interest in the innovation, including learning more about the innovation 
and it’s requirements for use. The personal stage indicates the individual is uncertain 
of demands of the innovation and has questions about his or her role in the use of the 
innovation while management includes issues of scheduling and organizing use and 
identifying available related resources. Consequence entails the outcome of use by the 
students and the changes involved to increase use of the innovation, and the final two 
stages, collaboration and refocusing, consider cooperating with others in regards to 
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using the innovations and exploring benefits or possible changes and replacements to 
improve tasks related to the innovation. 



Impact 



Task 




Self 

Information 

Unrelated 




Refocusing 

Collaboration 

Consequence 

Management 

Personal 

Awareness 



Figure 5 . Stages of Concerns about the innovation (taken from Hall & Hord, 1987, p. 
60) 



Levels of Use 

The second diagnostic tool involves Levels of Use. These levels represent a 
behavioral approach that indicates eight stages at which an innovation is being used: 
(0) nonuse, (1) orientation, (3) preparation, (4) mechanical use, (5) routine, (6) 
refinement, (7) integration, and (8) renewal (Hall & Hord, 1987, p.84). The purpose is 
to describe how the user moves from little or no knowledge of the innovation 
(nonuse), to introduction (orientation) preparation, mechanical use, and developing 
routine use. At this point, the individual has developed a full awareness of the 
innovation and is prepared to make changes in its use (refinement), to combine ways 
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of using it (integration) and to seek modification of the innovation (renewal) for 
increased use. 

Similarities and Differences in the CBAM and Rogers’s Models 

The CBAM model explains how an innovation is being used by an individual 
and at what stage it becomes part of routine use, whereas Rogers’s model emphasizes 
the decision-making process to implement an innovation. Both models demonstrate 
similar outcomes. For example, awareness and knowledge are both stages of 
introduction to an innovation, and informational, personal, and persuasion all indicate 
some form of use and influence from the change facilitator and further analysis of the 
innovation. Management, consequence, and decision are phases in which the 
individual identified the impact of the innovation by users; however, Rogers’s model 
suggests a decision to adopt the innovation has been made rather than focusing on 
consequences. The final stages, collaboration and refocusing, are identical to Rogers’s 
model (implementation and confirmation) because each indicates the innovation is 
being fully utilized. 

Although these models indicate similarities (Figure 6), differences point to 
concerns related to the innovation and the decision-making process. The CBAM 
model lacks explanation of how an innovation is being adopted, whereas, Rogers’s 
model demonstrates how the decision process works and possible effects of 
implementing an innovation. Furthermore, the CBAM model identifies an individual’s 
behavior during a level of use of the innovation, which does not reflect the outcome of 
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Rogers’s innovation decision process, which is to reach the decision to adopt and 
implement the innovation. In the end, as the CBAM model indicates, an individual 
may exhibit behaviors associated with a level of use during the adoption process, but 
not all individuals move through the adoption-diffusion status until comfortable with 
the innovation, thus reducing the success of the innovation diffusion process. 
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Figure 6 . Similarities of CBAM and Rogers’s models. 



User-Oriented Development Process 

Burkman (1987) reinforced Rogers’s view that adopters of instructional design 
products go through five stages as they pass through first knowledge of a product to 
using it (knowledge, persuasion, decision, implementation, and confirmation). 
However, Burkman indicates that there are two problems related to the utilization of 
these products: (a) convincing individuals to adopt and properly implement 
instructional materials and (b) convincing organizers to adopt products and getting 
instructors to use them properly (p. 435). With this in mind, Burkman developed a 
model supporting Rogers’s innovation decision process, described as the user-oriented 
development process. This process includes five steps: (1) identify the adopter, (2) 
measure the adopter's perceptions of the innovation, (3) develop a user-friendly 
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product, (4) inform the adopter about the innovation, and (5) provide user support 
(p.440). The model first identifies who will be affeeted by the innovation, and then 
measures the perception of the adopter towards the innovation, including identifying 
attributes that are perceived to be important. In the third step, the designer focuses on 
developing a product based on values considered important to the adopter, including 
perceived criteria that would ultimately lead to adoption. The final steps inform the 
adopter of the innovation’s valued attributes and provides post-adoption support. 
Burkman adds that attributes and characteristics of the innovation (relative advantage, 
compatibility, complexity, trialability, and observability) are key components that help 
determine the perceptions of the potential adopters related to the instructional product 
(P-442). 

Although both models are based on the innovation decision process, each 
model has a different agenda. First, Burkman’ s model was developed for instructional 
designers to analyze ways in which their innovations can be adopted. It is concerned 
with identifying strategies that would give an innovation the edge when it comes to a 
decision to adopt. Burkman’ s model is useful when utilized in the innovation decision 
process as a means to inform the adopter of the advantages relative to the innovation 
(knowledge and persuasion) but falls short of explaining how the model lends itself to 
the process of diffusion and implementation. In essence, Rogers’s model presents a 
holistic picture of the innovation decision process including the end result of 
implementation and confirmation, whereas Burkman only eonsiders the designer and 
perceived values to the adopter. And although Burkman’ s model exhibits 
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characteristics related to the innovation decision process, it falls short in explaining 
how the model contributes to the entire decision-making (selection), adoption, and 
diffusion (implementation) process. 



Change Process 

The concept of change is defined as “any planned or unplanned alterations in 
the status quo, including organizational change intended to influence an organization 
or larger system, an organism, situation, or process” (Lippitt, 1973, p.37). Change 
efforts are often used to change attitudes, change behaviors, or to change attitudes and 
behaviors (Zaltman & Duncan, 1977, p.26). 

The three key elements in the change process involve change agents, change 
objects, and change target systems. Rogers (1995) defines a change agent as “an 
individual who influences clients’ innovation-decisions in a direction deemed 
desirable by a change agency, usually professionals with university degrees in a 
technical field” (p.27). Change objects are the innovations involved, and change target 
systems are units which the change agent is trying to alter (Zaltman & Duncan, 1977, 
P-29). 

Historically, educational change consisted of innovations related to the 
development of curricula, implementing a process, incorporating ideas, and utilizing 
textbooks (Hord, 1987, p.l6). But over the years, it has transitioned to include 
technology. Once seen as part of modern classrooms and curriculums and primarily 
teacher- student based (Surry & Ely 2001), the educational technology change process 
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now consists of the stages of adoption, diffusion, implementation, and 
institutionalization. The following is a discussion and a comparison of models as a 
contributing factor to the adoption and diffusion process. 

Rogers’s Model as a Conceptual Framework 

The models described, Zaltman and Duncan’s (1977) resistance/adoption 
model. Hall and Hord (1987) concerns based adoption model (CBAM); and 
Burkman’s (1987) user-oriented instructional development process, all add to the 
literature related to decision making and the process of diffusion. However, Rogers’s 
framework emerged as most related to this study. The resistance/adoption model 
overlapped Rogers’s model, providing an explanation of stages useful for the 
beginning and end process of decision making, but failed to offer solutions for 
implementation. The CBAM model delivered an explanation of stages of concerns 
related to adopting an innovation but was targeted mostly to teachers and their 
concerns for adopting an innovation. The user-oriented instructional development 
process contributes information about the perceptions of potential adopters, but is not 
directly concerned with the outcomes described in the innovation decision process. 

Since this research is concerned with the evaluation, selection, adoption, and 
diffusion of a course management system, Rogers’s framework seemed appropriate as 
a conceptual framework and is most relevant to the central question involved in this 
study: “What are the factors that guide the decisions in selection and use of a CMS 
within small liberal arts institutions?” 
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Furthermore, Rogers’s innovation decision framework is a reflection of the 
process of decision making and the process of change, as is the case when selecting a 
CMS. And not only is Rogers’s framework useful in helping to explain the process of 
selecting a CMS, but also other technological innovations, as evidenced in the models 
described and other studies related to the innovation decision process. Liao (2005) 
examines the adoption and contribution of a web-based course management system at 
a college campus. West, Wauddoups, and Graham (2006) describe the experiences of 
instructors as they are persuaded to adopt a CMS and integrate it within their 
teachings, and Eineke (2004) analyzes the adoption and implementation of online 
professional education strategies by corporations. 

Summary 

This chapter reviewed literature related to course management systems, the 
diffusion of innovation theory and relevant models associated with the development of 
this theory. The diffusion of innovation theory is used to explain the process of 
adoption and diffusion while other models such as Zaltman and Duncan’s (1977) 
resistance/adoption model; Hall & Hord (1987) concems-based adoption model 
(CBAM); and Burkman’s (1987) user-oriented instructional development process all 
contribute to Rogers’s explanation of the diffusion of innovation theory. 

Rogers’s model, however, reflects a clearer explanation of the evaluation, 
selection, adoption and diffusion of a CMS. Further research is needed, however, to 
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provide additional literature to help explain the evaluation, selection, adoption, and 
diffusion of technological innovations. 




CHAPTER 3 



METHODOLOGY 

This study utilized the diffusion of innovation theory deseribed by Rogers 
(1995/2003) as a framework to examine faetors related to the proeess of evaluating, 
seleeting, adopting and diffusing a eourse management system. This study specifieally 
foeused on participants’ experiences related to the adoption and diffusion process of 
an innovation, specifically, a CMS. 



Rationale for Methodology 



This study describes the experiences of participants during the decision- 
making processes of evaluating, selecting, adopting and diffusing a CMS within their 
institutions. A qualitative case study method is specifically used as: 



a form of research undertaken on the premise that someone who is 
typically of a target population can be located and studied. In case 
studies, the individual’s history within an educational setting can be 
traced, growth patterns over time can be shown, functioning in one or 
more situations can be examined, and responses to one or more 
treatments can be measured. (Hittleman & Simon, 2006, p. 303) 



In this study, two institutions were included to examine and compare the 
process of evaluation, selection, adoption and diffusion of a CMS within an 
organization over a period of time. In addition, qualitative inquiry was used to 
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intertwine the relationship of the researcher and what is being studied, and highlight 
the meaning of the social experience being studied (Denzin & Lincoln, 2005, p.8). 

The data in this study encompass the description of participants while evaluating, 
selecting, adopting and diffusing a CMS. This approach allows the reader to hear and 
interpret the voices of the participants in the text (Hatch, 1995, p.l28). 

Research Methodology 

Researchers use qualitative studies to examine actions in a natural setting and 
look for factors that might provide insight into those situations (Vockell & Asher, 
1995, p.213). In order to describe the experiences of participants involved in the 
selection and adoption of a CMS, this study utilizes interviews to provide a holistic 
picture of the process. Merriam (1998) describes this as “an end product of a case 
study, which is an intensive, holistic description and analysis of a single instance, 
phenomenon, or social unit” (p. 27). 

Research Design 

This research is designed to answer the central question: What were the factors 
that guided the decisions in selection and use of a CMS within two small liberal arts 
institutions? In this study, a multiple case study approach was used. Two institutions 
were chosen and data from each institution were collected and analyzed separately. 
This approach allows data to be compared to find consistencies and inconsistencies 
between the cases. Overall, it provides an in-depth description from the participants’ 
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perspectives as they reflect on their involvement and experiences of evaluating, 
selecting, adopting, and diffusing a CMS. 

Description of Participants 

To identify possible participants within two small liberal arts institutions, a 
phone call was made to the Director of Technology of seven institutions within the 
Chicagoland and Northwest Indiana surrounding areas, to explain the purpose of the 
study. These directors were asked to participate in the study as an administrator along 
with identifying other possible participants (a combination of faculty and staff 
members). This communication focused on two criteria: selecting faculty, staff, and 
administrative personnel who participated in choosing a CMS and identifying early 
adopters of technologies, particularly individuals who adopted the institution’s 
selected CMS. The directors then forwarded names of possible participants and their 
e-mail contact information. E-mails (see Appendix A) were sent with an attachment 
(see Appendix B) to the suggested participants of each institution. These individuals 
were asked to respond to the e-mail only if they were interested in participating in the 
study. Two institutions each generated confirmation e-mail responses from five or 
more participants, while two other institutions generated three responses. The 
remaining three institutions generated only one or two responses. Since this study was 
designed to include at least six participants from each institution, five of the seven 
institutions with four or fewer respondents were then eliminated from participating in 
the study. Next, upon receiving confirmation e-mails from the institutions with six 
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respondents each, follow-up phone conversations were conducted to assess their 
availability to participate in the study. As a result, the two institutions were then 
selected to participate based on their availability of participants as described in the 
criteria. 

The participants were selected using a purposive sampling procedure, which 
allows researchers’ to choose whoever is available, and use judgment when selecting a 
sample based on prior information (Fraenkel & Wallen, 2000, p.l 14). 

Following this recommendation, two small private liberal arts institutions 
located in the Midwest with comparable enrollment ranging from 2,000 to 4,000 
students were selected to participate. These institutions were chosen based on 
proximity (within the Chicagoland and Northwest Indiana areas), similarity (private 
institutions), and type of student body (primarily an undergraduate campus with a 
small number of commuters). In addition, the participants of these institutions 
qualified for this study based on their direct participation in choosing their institution’s 
course management system. 

University #1 was founded in 1859 as a Lutheran institution. At the time of the 
study. University #1 had a current student body of nearly 4,000 students who came 
from most states in the nation and more than 40 countries around the globe. They 
pursued majors in more than 70 fields of study in five colleges, a graduate division 
and a law school. 

University #2, founded in 1837 by the Methodist Church, is a private, 
selective, coeducational, residential campus with approximately 2,400 students, at the 
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time of the stud, and a faculty-to-student ratio of 10:1. The institution offers over 40 
undergraduate Bachelor of Arts degree majors in the arts, humanities, sciences and 
social sciences. In addition, three degree options are offered within the School of 
Music. 

Six participants from each institution were selected from the categories of 
faculty, staff, and administration. Selected participants were chosen to provide 
different perspectives based on their direct experiences with evaluation, selection, 
adoption, and diffusion of their current CMS. Each participant was asked to describe 
their roles and experiences during the evaluation, selection, adoption and diffusion 
processes of their institution’s CMS. 

The faculty members from University #1 are referred to as Jim and Nancy. Jim 
was a tenured chair of his department, as was the case with Nancy. Staff members are 
referred to as Ruth, Janice, and Bob. Ruth was a full-time staff member and adjunct 
faculty in English. Janice was a technology support person, and Bob was a network 
administrator. The administrator, referred to as Al, was the Director of Technology. 

University #2 faculty members are referred to as Dan and Judy. Dan was a 
tenured faculty in Biology and Judy split her responsibilities as a faculty member, and 
Coordinator of Technology in Music. The staff members are referred to as Ron, Cathy, 
and Cynthia. Ron was formerly a technology specialist and Blackboard coordinator, 
and has moved on to another institution; Cynthia moved into that position. Cathy was 
the Associate Chief Information Officer of the Technology Division. The 
administrator, referred to as David, was the Director of Technology. 
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Data Collection, Procedures, and Sources 
To address ethical issues, all participants were informed of the intent of this 
study through a written consent form (Appendix C). Participants were also assured of 
their confidentiality upon written confirmation. After receiving the signed consent 
forms (Appendix C), phone and face-to-face interviews were arranged and conducted 
over a three-month period. Documents from University #1 (i.e., request for proposal 
cover letter and request for proposal) (Appendix E) were gathered as secondary 
sources and used to develop questions for interviews, particularly related to their 
experiences of evaluating the CMS. A survey (Appendix F) was collected from 
University #2 and included in the discussion and recommendations in Chapter Five to 
address how University #2 planned to evaluate and upgrade or discontinue their 
existing CMS. 



Data Analysis 

To ensure the consistency and validity of collected data, this study utilized a 
technique Miles and Huberman (1994) describe as a cross-case comparison. A cross- 
case comparison is done by forming types of groups and analyzing cases within those 
groups to seek common patterns or configurations (Miles & Huberman, 1994, p.l74). 
First, a single analysis of each case was conducted, and then all data were compared 
from the selected cases. This approach is recommended to increase generalizability, to 
discover processes and outcomes across many cases, to interpret how each case is 
significant to the study, and to develop more detailed descriptions and explanations 
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(Miles & Huberman, 1994, p.l72). This study also utilized a method called 
triangulation. Triangulation involves assessing printed documents, interviews, and 
observations as qualitative data to confirm findings (Lancy, 1993, p.20). Triangulation 
helps to minimi z e problems with internal validity (Merriam, 1998, p.204). Validity 
refers to the authenticity or truthfulness of the data (Neuman, 2003, p.l85). 

In this study, multiple data sources were used to triangulate findings. For 
example, the RFP (Appendix E) developed by University #1 was used to ask 
participants of University #1 specific questions concerning their experiences during 
the evaluation processes. The RFP also helped to generate responses from participant 
interviews related to the selection processes. In addition, member checks were 
conducted to establish credibility. Member checks involve describing the outcome of 
data with participants to clarify findings. This is done to enhance the validity and 
authenticity of data. Member checks were incorporated by first reading over the data 
collected from each participant at University #1 and University #2 and picking out key 
information such as dates and incidents emerging from the data. To verify dates and 
clarify experiences as described within the data, a follow-up phone interview was 
conducted with all participants. 



Organization of Data 

A handheld audio recording device was used to record the interviews. These 
interviews were then downloaded into a voice-editing software program and exported 



as an audio Wave format file. The voice-editing program then allows the user to 
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designate folders to organize and place audio files within their respective folder. In 
order to organize the Wave files, these files were given participants’ names and placed 
into their respective faculty, staff, and administration folders. These interviews were 
transcribed using HyperTranscribe, a computer-assisted qualitative transcription 
program by ResearchWare. I chose this software program because it facilitates the use 
of various forms of media files such as AVI, WAVE, MPEG, and QuickTime video 
formats to manually transcribe into RTE (rich text files) and documents. This program 
was particularly useful because of its visual coding feature, which highlights re- 
occurring words and phrases so that the user can identify patterns and place them into 
categories and themes. 

After transcribing the interviews, a coding framework was generated according 
to the research question and subquestions: What are the factors that guide the 
decisions in selection and use of a CMS within two small liberal arts institutions? 

1 . What were the issues leading to the need for a CMS? 

2. What kinds of procedures were used in the evaluation process? 

3. What were the roles of the administration, staff, and faculty in the 
evaluation and selection processes? 

4. How was the CMS selected by the organization? 

5. How is the CMS being used in the organization? 

The subquestions were used to guide the study and provide a thick description 
of the experiences of participants using data collected concerning issues, procedures, 
roles, and description of how the CMS was selected and used within each institution. 




40 



The coding framework included the following headings: evaluation, selection, 
adoption and diffusion. This was done to identify and classify themes that emerged 
within each part of the framework. 



Coding 

The next step incorporated a system called “coding” used to render and 
organize information about people, places, or events in a case setting (Creswell, 2003, 
p. 193). The coding process used the code and retrieve function found in the 
HyperResearch Qualitative Data Analysis Software (QDAS). This software is useful 
for separating text into segments or chunks, attaching codes to the chunks, and finding 
and displaying all chunks with a given code and placing common themes into similar 
categories. 

First, the transcribed data from each category (faculty, staff, and 
administration) were assigned the coding framework of elements: evaluation, 
selection, adoption, and diffusion. 

The data collected within each coding framework element was then analyzed 
to search for keywords used as themes (Appendix G) throughout each case. 

Analysis of Coded Data 

After reviewing codes generated from the CAQDAS program, the transcribed 
data were reviewed a second time to incorporate a process called inductive analysis, 
highlighting patterns, themes, and categories (Patton, 2002, p.453). The themes 
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(Appendix G) emerging from each category of participants (Faculty, Staff and 
Administration) were placed in each corresponding framework of elements 
(Evaluation, Selection, Adoption, and Diffusion). Next, a comparative approach was 
used for a cross-case comparison of data from participants in both institutions. To do 
this, each category (Faculty, Staff, and Administration) in the framework of elements 
was analyzed separately and then compared across each case. For example, data from 
University #1 under the Faculty heading (evaluation, selection, adoption and 
diffusion) was compared to the data from University #2 under the same heading. The 
data were then analyzed to search for different descriptions in experiences and 
common themes, particularly during each phase (evaluation, selection, adoption, and 
diffusion). This was done to describe the differences and similarities found between 
data sets and provide a thick description of the cases involved. Next, participant 
narratives from each category (Faculty, Staff, and Administration) were placed in 
reference to the subquestions and then presented in a descriptive narrative (Chapter 
Four). 



Summary 

This chapter covered the methodology of this study, research design, and the 
collection, analysis, and organization of data. Chapter Four provides a description and 



analysis of data, generated from the framework of elements. 




CHAPTER 4 



FINDINGS OF THE STUDY 

This chapter discusses the findings within this study. The data were organized 
using a framework of four main elements: evaluation, selection, adoption, and 
diffusion. The study included six participants each from two institutions (two faculty 
members, three staff members, and one administrator). The findings are presented 
within the four main elements. Each element presents findings from the participant’s 
perspective, guided by the research question and organized by subquestions. 

Research Question 

What are the faetors that guide the decisions in selection and use of a CMS 
within two small liberal arts institutions? 

Following Miles and Hubermann’s (1994) suggestion, the findings from each 
case are discussed separately, followed by a cross-case comparison discussion of 



findings. 
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Part I: University #1 
Evaluation 

Each of the individual groups (Administration, Staff and Faculty) will be 
discussed separately after each research subquestion. In addition, a summary of each 
framework element (evaluation, selection, adoption and diffusion) will conclude the 
discussion of each sub-question. 

What Were the Issues Leading to the Need for a CMS? 

In the early 1990s, University #1 focused its commitment to education on face- 
to-face communication. The first form of an on-line communication tool incorporated 
the use of a server and web browsers such as Netscape and Internet Explorer to place 
files on the Internet. Participants of University #I indicated that the institution did not 
support these tools because University #1 at that time focused on delivering courses 
via face-to-face. 

While attending a conference, Nancy, a department chair in Education, was 
introduced to the concept of a CMS. With an interest to utilize technology to improve 
her pedagogical practices, Nancy introduced the concept to others, which led to the 
development of a small group of faculty interested in using technology in their 
courses. In 1999, one of the faculty members received a grant to purchase the first 
adopted system, TopClass, designed by WBT Systems. Not many of the faculty 
community in University #1 adopted TopClass, and there was very little effort to 
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support, manage, or coordinate workshops on how to use the system. The institution 
recognized this and eventually developed a committee to evaluate, select, and 
recommend a new system to serve the needs of the faculty and the institution. Also, a 
cover letter and a request for proposals (RFP) (Appendix E*) were sent out to vendors 
in the spring of 2000 to announce the institution’s intentions to purchase a new 
system. That process led to the adoption of Prometheus, developed by faculty 
members of George Washington University. 

An e-mail (Appendix D) was sent to faculty to announce the adoption of 
Prometheus in February of 2001. But in 2002, Blackboard acquired Prometheus and 
the institution again chose to select a new CMS, which led to developing a second 
RFP (Appendix E). The institution eventually settled on Blackboard and began placing 
all courses offered by the institution on Blackboard. Faculty then had the choice of 
using Blackboard either to develop a full on-line course or as a supplement to face-to- 
face instruction. In addition, to move beyond traditional face-to-face instruction, the 
university offered additional financial incentives during the summer, a time in which 
most faculty members have more time to dedicate to develop on-line courses. The goal 
was to then offer the on-line course in the upcoming academic year. 



^ There were two RFPs sent to vendors. The first was created in 2000, and the 
second in 2002. Both RFPs contained the same information, the 2002 RFP was 
slightly modified with updated information. 
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What Were the Roles of the Administration, Staff, and Faculty in the 
Evaluation and Selection Processes? 



Administration . When University #1 decided to transition from 

TopClass into another CMS, Al, the head of the Technology Department, was 

given a directive from the provost’s office to evaluate a new system. Al was 

the only administrative staff member to participate in the evaluation process. 

Al commented that during the first evaluation process, both Prometheus and 

Blackboard were evaluated, with Prometheus being selected as the new CMS. 

In the fall of 1999, the provost’s office appointed a task force to see what 
CMS we should have here. But there were a number of people weighted to 
the faculty, not administration, on the task force. It is my feeling that when 
we select a system we should see what is viable for the institution. You look 
at it like a marriage, a long-term investment with a lot of faculty putting effort 
into it. There were concerns with support, networks, servers, etc. We went 
through that process and it was interesting that one vendor, Prometheus, 
seemed to meet the needs of the faculty because they were a faculty-oriented 
vendor. 



However, University #1 was notified by Blackboard concerning the takeover 
of Prometheus, which made some faculty members unhappy. Thus, another evaluation 
and selection process began. 



When Blackboard took over, people were not happy. They liked 
Prometheus, and you knew they were going to drop Prometheus. It was 
introduced through an e-mail that they [Blackboard] were going to take 
over. We began looking at open-source systems like MIT that was free 
but you had to have support, and there were things missing like a grade 
book and faculty were not willing to give up that pedagogical feature. 
Practically immediately we decided to do the REP over again. . ..One of 
the things the committee did was to have a study group that was 
instrumental in re-issuing the REP. But the evaluation was more 
intensive and some who submitted responses lost advantages over time. 
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It was a very similar process. One of the things asked for was the 
ability for conversion from Prometheus to the new system. We dealt 
with that issue as aggressively and effectively as possible. 

Staff . Bob, a staff member of University #1 Technology Department, played a 
small role in the evaluation process. He reflected upon his role: “I was part of 
technical support and in charge of fielding questions from the committee concerning 
technical issues and requirements.” 

Janice, also a staff member of University #1 Technology Department, 
participated on the committee because her job responsibilities called for a user support 
person to train faculty, staff, and students to use software. Janice had no prior 
experience with a CMS and trained herself to use TopClass. Janice described her role 
in the evaluation process as an end-user and described the software as user-friendly: 



We all had an equal say and everyone had their own specialty area. It 
was a mixture of interests, and we looked at the technical end of how 
we were going to serve everyone. I was looking at the back end on how 
to present this to faculty and how user-friendly it was to get it on board. 



Faculty . Jim, a faculty member and chair of his department, considered the 
evaluation process for Prometheus as a group-oriented task, not a process. Jim 
explained his role in the evaluation process: 



I saw my role as two-fold. I had a course running and if we had to 
change software, I wanted the best one that would accomplish the 
objectives and be easy to use. Second, I was a committee person as an 
early adopter. But at the same time I knew the university was not 
buying it for me, they were buying it for all of the faculty. 
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Nancy, also a faculty member and ehair of her department, was one of the first 
to initiate discussions concerning CMS. Nancy described her role in the evaluation 
process: 



I was a leader at first but a behind-the-scenes leader because I was 
aware of how many people on campus would use it and conneet with 
our elasses. The thing I was concerned with was what pedagogieal 
things could it do to help me as a faeulty member that I wanted to 
aeeomplish? 



Ruth taught a course for the English Department and deseribed her participation in the 
evaluation proeess as a “volunteer” on the eommittee for the Prometheus evaluation 
process. 



I volunteered on the committee, and I was interested in it. At that point 
I was teaching a course for the English Department and I thought it 
would be niee to have some say on the CMS since the previous CMS 
was unsatisfactory and chosen by others. Beeause my job involved 
working with software as it relates to teaehing, it seemed like a good 
thing to be on the eommittee. 



Ruth described the evaluation procedures as a long, yet dedieated proeess: 



I remember it was a long and torturous proeess and I was not prepared 
doing this type of researeh. I had not been in any group where people 
work out any request for proposals. I thought you chose some company 
that other institutions used and you send out an REP, but I did not 
realize how eomplex the REP had to be. I realized it was a mueh bigger 
task than I had antieipated. I was impressed with how dedicated the 
members were, particularly faculty members. It surprised me how they 
took an interest in some of the things we were looking for. 
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Summary 

Each participant played a critical role in the evaluation process. For example, 
Jim, Naney, and Ruth all provided a faculty perspective and their personal experiences 
of using Prometheus and Blackboard. Jim was concerned with how easy the system 
was to use, while Nancy focused on pedagogical features. Bob and A1 both offered 
technical and administrative support. Janice, however, viewed the experience as an 
end-user and wanted the system to be user-friendly. Overall, each participant’s unique 
contributions played an important role in the evaluation proeess. 



What Kinds of Procedures Were Used in the Evaluation Process? 



Administration. Al, the administrator, discussed some of the procedures used 
to evaluate eaeh system: 



The faculty took the first initiative. They prepared an REP for course 
managment systems. We looked at a lot of sources and compared the 
major systems. We looked at features within the CMS and put together 
an REP and posted it on the web. Four or five revisions were done 
before it was issued about a year later because we wanted to be sure all 
needs were refleeted and the proper people had input. It was a very 
collaborative effort to put together the REP and evaluate the responses. 



Staff . Bob, a staff member, became a member of the committee through job- 
related responsibilities as one of the university’s web masters and web administrators 
Bob had previous experience with placing the prior system (TopClass) on the server 
and participated in evaluating the technical requirements and support issues for the 



new CMS. Bob recalled the process as overwhelming, yet needed: 
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The RFP was edited and sent out to about 9-10 vendors. TopClass, 
WebCT, Prometheus, and Blackboard were key players. We had to 
look at system requirements and see if we had to upgrade, what hard 
drive space was needed, and also, we evaluated if we could integrate 
mail into the system, and was there a calendar integrated into the 
system. 



Bob described the second evaluation process, which led to selecting 
Blackboard, as being easier because the previous RFP provided a blueprint to start the 
evaluation process. Bob indicated the process took less time: 



The RFP was not a new process. We took the old one and tweaked it 
based on new technology concerns. We sent out the RFP based on a 
deadline, and we would see what's available and check out other 
schools to see what's out there. We ended up evaluating Blackboard 
because the transition process was easy, the professors liked it, and it 
met systems requirements, etc. My plan was the same, but I had 
thoughts of here we go again, but it was necessary. With Prometheus 
we got more people on line and with Blackboard we could get to the 
next level. In addition, we knew that Blackboard ran Prometheus for 
about a year before our RFP went out. 



Faculty. Jim was the chair of his department. Jim described the evaluation task 
that led to Prometheus as fairly easy: 



They [the university] wanted us to evaluate and pick the one that was 
most promising, so we had a meeting where we chose websites and 
looked at information about each vendor’s CMS. Then the university 
asked these vendors if they were interested in picking us up as a client. 
Next, they sent out these questions called an RFP and each had to 
respond and answer all the questions the administration and faculty had 
about their CMS. We then met with the vendors and looked at the 
products, asked them questions and tried to load our content onto their 
systems to see how it worked. It was a unique and thorough process 
and we knew that it was going to be expensive. 
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Jim considered the Blackboard evaluation process as easier, having previously 
participated in the Prometheus process. 



In order for the venders to answer questions, people in the room had to 
be people who could ask the questions. At that stage, the university 
used a committee of people who were experienced because others 
would have been lost. 



Nancy, a tenured faculty member in the Department of Education, became part 
of the committee because of her personal interest in teaching with technology and 
being challenged to look for ways to incorporate technology in classes. When 
describing the evaluation process of the first CMS (Prometheus), Nancy indicated: 



I thought it was a great process because we had an awful lot of research 
with books, and then we had to rate these by spreadsheets, 250 pages of 
components we had to look at and then the data was presented to the 
group. It was a lot of work and I felt good about the work put into it. 
Many sections I could not understand because they were technical 
issues I had no idea about and I was not concerned with. 



Summary 



Similar procedures were used to evaluate both Prometheus and Blackboard. An 
REP was generated to solicit vendor responses in both evaluation periods, and Al, an 
administrator, considered the REP an integral document used in the evaluation 
procedures. Jim and Nancy, both faculty members, indicated each evaluation process 
appeared easy. In addition, as Bob recalled, in order to evaluate each system, costs had 



to be considered. 
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Selection 



How Was the CMS Selected by the Organization? 



Administration . A1 represented University #1 as the administrator, Bob 
participated as a staff member, and both Jim and Nancy participated as faculty 
members. Al, the head of the Technology Department, viewed selecting Prometheus 
as a clear process: 



Blackboard was offering several options for five thousand dollars a 
year, but it was light; Prometheus was twenty-five thousand dollars, 
and the faculty was sold on Prometheus. Blackboard offered options 
that were higher prices like thirty thousand dollars, so on the surface 
Prometheus looked good. The faculty liked Prometheus, in part because 
it was designed by faculty. 

However, the institution expressed more caution when selecting 
Blackboard. Most were familiar with the selection process and being a 
year and half before, so there weren’t many changes except the things 
faculty recommended. The committee did not want to just go with 
Blackboard, but they were in agreement. I then passed the decision 
along to the Teaching and Learning Technology roundtable, the 
provost’s office, and the finance office. 

Staff . As the system administrator. Bob indicated that the decision to select 
Prometheus leaned heavily on initiatives such as cost and technical support: 



It was a top-down selection and we had some initiatives to select from 
such as cost and compatibility. The final selection of Prometheus 
technically dealt with compatibility with upgrading easily and backing up 
the system. 



When the sale of Prometheus was completed. Bob recalled the differences in 



communication patterns, which affected the decision to select Blackboard: 
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There was quite a bit of difference in support, such as contacting one 
person when it was Prometheus, but now Blackboard has a bigger staff 
and you have to go through a couple of people to get things done. Now 
the problem is that the technical support people go through a revolving 
door and as soon as we get comfortable, someone leaves. I pretty much 
dealt with the same Prometheus people the entire time. 



Bob was also concerned with how easily the CMS could be rolled out. The 
other concerns were costs and support, how expensive it would be to maintain a 
system, including how many extra employees would need to be hired and would there 
be continuous support by the company. 

Faculty . The selection process for Prometheus was slightly different from 
Blackboard. Jim, a department chair, indicated that the Prometheus selection was more 
of a process: 



We all got to vote in the end, and asked which one did you want? We 
all concluded our choices and Prometheus was the outstanding one, 
with pro/cons. The technology Department was looking at support, the 
university was looking at costs, and we were looking at faculty issues 
such as would it help us to teach easier. 



The Blackboard process became easier because of the experiences learned 
from the previous process. 



As I became more familiar with the process, and began to talk with the 
vendors, I learned more about CMS's. So I started improving my own 
ability to look for important things that I knew would affect me and I 
had a big interest in this. We all did because whatever system we chose, 
if there were reservations any of us had, we needed to get this in the 
open and needed to hear the answer from vendors. 
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Nancy, a tenured faculty chair in the Department of Education, indicated her 
experience with the Prometheus selection process was time consuming and positive: 

This was an institution-wide decision now and they wanted more 
faculty to get involved using a CMS. It was a long process if you did 
not like attending meetings, but my voice was heard because I had a 
vote. A pretty positive selection process because faculty were the ones 
to use it. I think voting was done separately through spreadsheets and it 
became a consensus. 

Blackboard presented a few problems during selection process, but the 
procedures were similar. Nancy indicated that the experience of working with the 
committee was exhaustive and tiresome, yet pleasant: “It was another long process but 
we had to make the decision on Blackboard.” 



Analysis of the Evaluation and Selection Process 



University #1 had the opportunity to select a new CMS twice within a two-year 
period. That led to a rigorous process of evaluating and selecting a second and then a 
third system. Eaculty committee members described the second process as easier, 
having previously seleeted another system, while the staff and administrators viewed 
the process as a “process” but also added that revising the REP helped make the 
process easier each time. Bob suggested that one of the deciding factors leaned toward 
the costs of each system. Jim suggested most of the committee members were 
comfortable with the pedagogical features each system had to offer. Overall, each 
participant felt comfortable contributing to both evaluation and selection processes 
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because they were able to work with practically the same committee and had gone 
through the same process previously. 



Adoption and Diffusion 

How is the CMS Being Used in the Organization? 

Administration . A1 participated in the adoption and diffusion processes as the 
administrator while Janice participated as a staff member. Jim and Nancy both 
participated as faculty members. Al, the administrator of the Technology Department, 
discussed how the university community began to adopt Blackboard: 

We had a fair number of faculty members who adopted Prometheus. 

The major thing that helped adoption was incentives from provost’s 
office such as on-line course development, and then you get a mass of 
people using it and students begin asking other instructors why don't 
they use this and one faculty mentors the other, so it spreads faster. 

Al, the head of the Technology Department, suggested the diffusion process 
was not as difficult as anticipated: “As far as the faculty was concerned it was doing 
well and we thought they had shortcomings that could be fixed.” 

Staff . Janice, a Technology Department staff member, described the adoption 
process of Blackboard as challenging because many of the faculty exhibited caution to 
moving to another CMS after using Prometheus: 



To get them on board with a second software was not easy. Some did 
not want to learn because they were concerned if this was bought out, 
they did not want to learn new software. 
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Bob recalled the adoption and diffusion processes of Prometheus created a 
sense of fear amongst faculty members. However, the Blackboard processes were an 
easier transition because of the previous experience with Prometheus. 



One of the issues was that the utility was not in place to move TopClass 
to Prometheus, so that was an anxiety the professors experienced. It 
was messy and was not easy to do. We walked on eggshells for a while. 
However, the training helped us to move people from each system. 

With Blackboard, it was easier because we had tools to transfer 
courses, the installation was easier. . .same system requirements, ease of 
use, and transitioning courses. Blackboard was installed within 3 days, 
which is normally a two- week process. 



Bob commented that the support provided by Blackboard made it difficult to 
create strategies for diffusion when compared to the Prometheus process: 



George Washington University still wanted to run Blackboard after the 
acquisition, but our contacts were severed with George Washington and 
went to Blackboard. There was quite a bit of difference in support. 



Janice, a user support staff personnel, recalled the diffusion of Blackboard was 
a cautious move and created some hesitation amongst faculty to adopt Blackboard: 



I'm used to software being bought out and phased out frequently, but 
faculty was not used to this. To get them on board with Blackboard was 
not easy. Some did not want to learn because they were concerned if 
this was bought out, they did not want to learn new software. 



To adjust, Janice created a “team” of heavy users and encouraged them to 
showcase their courses to others in order to influence adoption. In addition, the 
administration offered gift certificates to participants who attended the workshops, and 
stipends were provided as an incentive to design summer courses on-line. 
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My job was to figure out how to stay on-line with summer courses and 
retrain faculty that had been using Prometheus, plus add more faculty. I 
ended up going to my hard hitters, my big "8" users who used it 
well. . .The big "8" represented all five colleges, and it was a mentor for 
each building on the campus. I taught the big "8" first and asked them 
to evaluate my instructions because it helped me write instruction 
better. They got their courses up first, I then opened instructions to 
others, and asked them [the big “8”] to come in one by one to be a 
guest speaker so they could share their course with other faculty that 
were being trained, it was a win-win for everyone. The institution also 
provided stipends to create courses on-line, offered training sessions 
with a 25 -dollar certificate for participating, and provided lunch during 
training sessions. 



Faculty . Jim, as a department chair, approached the Prometheus adoption phase 
with caution: 



I remember I was nervous because I run into problems and would have 
to ask the Prometheus support center what to do and I would have to 
create work around this. There were periods where Prometheus would 
collapse and I had to go back to using the university web server. That’s 
why I kept files in the old-fashion way. 

Jim explained that adopting Blackboard was an easier process, with 
very little caution. 



In some ways I have more control in Blackboard. I did not have any 
issues with Blackboard and could do it in my sleep. But you have to 
watch out for when you want things to appear. 



The experience of adopting both Prometheus and Blackboard was different for 
Nancy as a chair of the Department of Education. As one of the early adopters, Nancy 
became a trainer, thus she experienced the adoption process from a trainer’s 



perspective rather than an adopter. 
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I had a different view of the adoption process. The university had 
training sessions but there was reluctance amongst faculty to attend. I 
was one of the trainers. The workshops were held in the summer and 
tailored to the users. 

Jim also participated in the adoption and diffusion processes as a 

trainer. 



I don't think it was mandatory but we had workshops to learn. They 
were sign-up workshops if you were interested and when a few of us 
became more knowledgeable, they asked us to demonstrate the courses. 



Summarv of the Adoption and Diffusion Process 



A1 saw the adoption and diffusion processes as uneventful. Unfortunately, Bob 
and Janice described the processes differently. Bob had some difficulty contacting 
Blackboard personnel, and Janice felt the faculty exhibited a lot of caution to move 
from one system to another. Also, Jim approached adopting Prometheus with caution 
but felt comfortable with Blackboard, having gained more experience using a CMS. 
Workshops and incentives were not a part of the Prometheus process; however, 
offering incentives to attend workshops helped the adoption and diffusion of 
Blackboard. Fortunately, it was helpful for both Jim and Nancy to wear two hats as a 
trainer and a user, in order to influence others to use the system. Nevertheless, all 
participants learned from their experiences adopting Prometheus and approached 
adopting Blackboard with less caution. 
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Part II: University #2 

Each research subquestion will be analyzed separately. Each coding 
framework will be discussed separately, including an analysis of data from each 
participant. 



Evaluation 



Issues Leading to the Need for a CMS 



The first on-line CMS system was a class discussion tool called Discus, 
designed as a means for faculty and students to communicate on-line. Approximately 
35 faculty members adopted the system. As these faculty members became more 
sophisticated with on-line discussions, some expressed their frustration with wanting 
to do more using technology. Ironically, while attending a conference, Ron, a former 
employee of the Technology Department, was introduced to other CMSs and began 
learning a system called Web Course in a Box. He used that CMS to design a Spanish 
course on-line. 

In 1997, 1 went out to Arizona State University as an intern in the Scope 
Program and helped them use a tool called Web Course in a Box. I built a 
web base page to train them and after I returned to campus, we had a request 
to develop a Spanish course, so I did it with Web Course in a Box. 

My experience led to researching CMS’s to see what other schools were doing. 




59 



Judy, a faculty member in Music and a staff member in the Technology 
Department, recalled Discus as a discussion tool, which eventually outgrew its 
usefulness, and created problems for faculty to deliver documents to students. 



We found it [Discus] was not adequate to help faculty to push out 
documents and collect them. I remember not using it because it was a 
limited systems and hard for people to use. 



Dan, a faculty member in Biology, had been using Discus and was now 
looking to enhance discussions in his course: 



The goal was to use biology in different applications of science. The 
technology center was a place where faculty could go and accomplish 
pedagogical things with new technology and there were people who 
were pushing you to what you wanted to do and developing ways to 
accomplish it. 



More faculty members had similar interests, so as a result, the Technology 
Department began looking at what other institutions were using. In 1998, a grant was 
allocated to begin a yearlong process to evaluate and select a system. WebCT, E- 
College, and Blackboard were chosen as three of the leading vendors to evaluate. 

After evaluating several systems, the institution adopted Blackboard as a pilot 
CMS. The Technology Department acknowledged that some faculty members had 
some experience using a CMS but had little time to design courses. Thus, the 
Technology Department instituted an effort to connect the faculty who were using 
discussion tools with the faculty who were interested in creating a full on-line course. 
The experience was unique because the university had established a relationship with 
colleagues from another liberal arts institution that had adopted Blackboard and had a 




60 



number of people using the system. That institution collaborated with University #2 by 
providing Ron, a former employee of the Technology Department of University #2, an 
account and administrative access to their system during their evaluation period. A 
year later, the university adopted Blackboard as the institution’s CMS. 



What Were the Roles of the Administration, Staff, and Faculty in the Evaluation and 
Selection Processes? 



Administration . The process began similar to that of University #1, and David, 
the head of the Technology Department, described how the institution engaged in the 
evaluation and selection process: 



There was an RFP that included not a very detailed proposal but it 
talked about our educational aspirations and looking for a CMS that 
will help instructors accomplish XYZ and not specify things like 
servers. We focused on pedagogy and presented the RFP from an 
instructional standpoint rather than a technical standpoint. 



David, the only administrator of University #2 to participate in the study, and 
the head of the Technology Department, saw his role as one of the initiators to bring a 
CMS to campus: 



I had the opportunity to see all of the CMSs (WebCT, Blackboard, etc.) 
prior to the evaluation process and came to the table with the 
philosophy that the web was an important resource for instructors. We 
initially did a one-month pilot in the winter term with Blackboard, but 
it turned out to be a disaster because it could not connect to our campus 
and we decided to buy a test server, so we tried it again in the spring. 
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Staff. Ron, a former employee of the Teehnology Department, recalled his role 
during the evaluation period. Ron first networked with other liberal arts institutions to 
see what they were using and then began setting up Blackboard as a pilot CMS on the 
University #2 system and also on the system of another liberal arts institution. 

“We first set up Blackboard and conducted workshops, and at the same time, we 
created 15 test courses on another institution’s system to let the faculty evaluate.” 

Faculty. Dan, a faculty member in Biology, had consistently networked with 
the Technology Department to discuss ways in which he could use technology in his 
courses. Dan recalled: 



I brought to the Technology Department what I wanted to accomplish. I 
asked if there was a product that would allow me to accomplish 
different tasks. I was looking for a means of improving teaching in 
Biology. 



What Kinds of Procedures Were Used in the Evaluation Process? 



Administration . David, the head of the Technology Department, participated in 
the evaluation process as an administrator. Ron was not included as a staff member; 
however, Cathy did participate because of her job description. Judy was the only 
faculty participant. David, the head of the Technology Department, described the 
process as challenging because of issues related to support: 



It was about a year-long process in which we had faculty members 
coming in to get a feel of the features provided and how effective the 
tools would be. One of the key issues we were unhappy with was the 
support and reliability of Blackboard. 




We felt strongly about adopting a system that was reliable and that 
could be quickly fixed because you could not afford to lose time in 
class. 
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Staff. Cathy, a member of the Technology Department, described how the 
process incorporated the input of faculty: 



We found a few faculty members whom we partnered with to evaluate 
Web Course in a Box, and it was a learning experience with the 3-4 
faculty people. We found that it was a really young tool and it was not 
very useful, so Blackboard became prominent. 

Judy remembered the Blackboard representatives were aggressive in their 
presentations, and emphasized that the Technology Department leaned favorably to 
Blackboard: 



I remember that Blackboard was being aggressive about getting 
colleges and I remember them [the Technology Department] being very 
favorable about the Blackboard presentation, and economically it was 
more feasible, as it seemed to serve the different needs in its basic 
version at the time. 



Faculty . When reflecting on the responsibilities of being a combined faculty 
and technology person, Judy recalled looking at the process in a reverse role: 



I actually think I did it in the reverse; we talked to faculty of what they 
needed, it was all about what they wanted to do in their classroom, so 
my perspective was different because it was pedagogical such as 
remembering some people wanted images or other needs and how well 
it was going to fit with faculty. 



Judy also recalled attending a presentation by a colleague, which helped her to 



evaluate Blackboard: 
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I remember that a colleague from another liberal arts institution came to 
campus to talk about Blackboard. I remember her presentations and that 
she linked functions of Blackboard with instructional philosophy, like 
content presentation, and it was a pedagogically oriented presentation. 

It told me that Blackboard was thinking pedagogically as well as 
technically and it was appealing to me. 



Selection 



How Was the CMS Selected by the Organization? 



David, an administrator, and Cathy, a staff member, were the only participants 
in the selection process. David discussed how Blackboard was selected and funded: 



The university is fortunate with financial resources because the 
information resources group had money from year to year to spend. 
This included finding out what faculty needed, so it was not a president 
or provost decision, it was from a discretionary budget. We selected 
Blackboard because they looked liked they were going to be around. It 
was not a cost issue because we brought into the low level of 
Blackboard. 



Staff. Cathy, a member of the Technology Department, reminisced about how 
the final selection was made: 



We looked deeper and the price was right and we thought it was going 
to be useful and at that point we enlisted several faculty members and 
identified Blackboard as the replacement for Discus. 



Summary of the Evaluation and Selection Process 



The evaluation and selection process was slightly different for University #2. 



Faculty members became interested in using a CMS, which led to bringing vendors on 
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campus to present their systems to the institution. Faculty were also invited to 
workshops to test Blackboard and discuss their evaluation of the system. In addition, 
the evaluation process was ongoing as Blackboard was tested for a year. 



Adoption and Diffusion 

How Is the CMS Being Used in the Organization? 

Administration . David, the head of the Technology Department, participated in 
the adoption process as both an administrator and faculty member. David discussed 
from an administrative standpoint, some of the problems encountered in adopting the 
system: 



Support was an issue because it took a lot of time to contact someone 
when the system was down. I learned a single point of contact was the 
way to go, so I assigned a centralized help desk, and they knew who to 
call to solve the problem. 



In addition, David also taught a History course, and described how other 
faculty members embraced adopting the Blackboard system: 



I was one of the faculty members talking about what I was doing with 
History courses. When the instructors saw what I was doing, others 
seemed to follow, or a professor would talk about what they would be 
doing. For example, one English professor used to be a lecturer and 
now Blackboard allowed him to bring the class into discussions to 
expand the way he taught and it allowed students to come to class 
better prepared. 
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Staff . Ron was involved in the adoption process before leaving the institution. 
He recalled how the faculty began to adopt Blackboard: 



Most faculty jumped on wholeheartedly, and others did not want to 
change, so we took extra time with them to explain things. The growing 
student workers helped them by explaining how to put syllabus, chat, 
etc., onto Blackboard. 



Cynthia became involved after taking over Ron’s position. She recalled the 
process used to entice faculty to adopt Blackboard: 



From a faculty perspective we got it up and going and targeted the 
people who were using Discus in the early first semester and came back 
in the fall and phased out Discus. We did not migrate a lot of faculty 
until we got the Discus people over and made a cheat sheet to go along 
with the training to help them move along and then once we had them 
rolling we started a showcase event during lunch with a faculty member 
that has been doing things and that begins soliciting interest from other 
faculty members. The key characteristics were that we selected it and 
adopted a tool and made it available and described what it would do 
and had them make the decision to use it. 



The biggest task for Cynthia was to create a strategy to approach the adoption 
process, as if she had been involved all along. 



My strategy was to get together as a team and figure out the things we 
needed to work out, what issues we had for support, what we could 
provide them and who was going to provide support from the first level 
to the third level, also sitting down and making policies that worked 
with the goals of the university. You did not want to provide something 
for the students and faculty that is going to go against what the 
department is doing or what the institution is all about, so it had to 
balance with the vision of what the future would look like. 
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Faculty . Dan remembered how Blackboard helped him to improve his 
pedagogy in his Biology courses: 



The first class I used Blackboard in was paperless, whieh was 
interesting both for the instructor and the students. It demonstrated that 
when students are engaged in the topic they are willing to spend a lot of 
time outside of the class posting information and dialogues. 



As a technology administrator, David was in charge of overseeing the diffusion 
process. David suggested the process was a ehallenge and it helped to learn about the 
issues and address them in a timely manner: 



We ereated accounts for every eourse on campus whether you used it or 
not. We also provided support, but other faeulty members, the early 
innovators, also helped get others to use it. In addition, we created the 
tools to help faculty members talk about what they needed to 
aceomplish, so all talked with each other and got them excited about 
this. 



Ron, a former employee of the Technology Department, deseribed his 
experienee of rolling out Blackboard to the faculty: 



When we started to roll out Blackboard, it was a four-prong approach: 
training, policy (administrative, how to back up things, course ID 
numbers, ete.), support, and marketing (promote it to the faculty). I 
remember I would write weekly or biweekly updates so that the faculty 
could keep up with all of the four levels and also keep the players 
informed of what they were doing. 



Staff . Cynthia, a Technology Department staff member, discussed how 



workshops were incorporated into the diffusion process: 




67 



We did workshops and showcases, and from the maintenance side we 
created course shells for every course and enrolled all students and 
instructors so that they can use it if they wanted to. That made it easy 
because they did not have to request a Blackboard course, they could 
start using it. And shared folders for classes were also part of that. 



Orientations were also provided to students as requested by instructors. 

Cynthia added that students appeared to be more advanced and rarely needed an 

orientation. However, it helped the confidence of the faculty to request a workshop 

before using Blackboard in their courses. 

We would provide classroom-setting training for students as an introduction 
and over time they did not need this orientation and we recognized they did 
not need training, so we would do this as needed. It made the faculty person 
feel comfortable. As students became more familiar with using Blackboard, 
the orientations became very minimal, and we found they just needed help 
with things like digital drop box, logging on, etc. 

Judy, a faculty member in Music and also a technology staff member, recalled 
the diffusion process as sometimes troublesome: 



After adoption we had a lot of trouble with support, it would go down 
randomly because we had the basic model. I remember a couple of calls 
we made asking them what we can expect when it went down. . . it was 
just a response time. 



Judy also recalled how the technology staff incorporated workshops for 



faculty: 



We organized events such as Friday at 6, when we would get together 
and eat food, and trained them [faculty] on Blackboard. The summer 
sessions were week-long trainings and we did on demand, which meant 
going to classes. 
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Faculty . Dan, a faculty member in Biology, had problems placing files onto 
Blackboard and expressed concerns about packaging files and the time it took to 
prepare materials for courses: 



The one problem I had was that if you generated a document outside of 
Blackboard, it would ask you what was the file, and if the student 
cannot access it, you had to make sure the application (Word, etc.) was 
available on the computers where the students were accessing this. It 
also made it difficult to integrate video and sound clips. There were 
some limitations, but it opened doors. Also, the front end loading takes 
a long time to assemble everything and get it into a package to where 
the students can access things; that time frame takes more time than 
generating paper copies or handouts. 



On the other hand, Dan felt Blackboard was a useful tool for his courses: 



It worked too well; it took a tremendous amount of time on the faculty 
part because the students expected responses within a significant 
amount of time and expected a posting from us. It became clear that we 
needed to come to an understanding when they should expect 
responses, but it took up a lot more time than we expected, despite it 
was three of us and only 16 students, meeting twice a week. 



Summary of the Adoption and Diffusion Processes 



University #2 experienced problems with support from Blackboard and found 
that they needed answers as to why the system would randomly malfunction. Judy 
described that process as troublesome while David offered solutions. Dan, a Biology 
faculty member, had problems with sending and packaging files and became 
concerned with the time it took to prepare content for a course. In addition, Cynthia 
found ways to entice faculty to adopt Blackboard, which included a luncheon for 
faculty to showcase courses they had developed. To enhance the diffusion process, all 
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courses were populated with Blackboard so that faculty could easily adopt the CMS 
and training was provided upon request. Overall, the adoption and diffusion process 
was easier as faculty became comfortable using Blackboard. 

Cross-Case Analysis 
Evaluation and Selection 

University #1 was first introduced to the concept of a CMS when a faculty 
member attended a conference and became intrigued with the idea of newer ways of 
enhancing pedagogy through technology, particularly using a CMS. University #2 had 
been using an on-line discussion tool (Discus); however, a staff member also attended 
a conference, and became aware of CMSs. 

The process for both institutions began by generating an REP to solicit bids 
from vendors. University #1 developed its REP only after the decision was made to 
select the second and third CMSs. University #2, however, developed an REP when 
the decision was made to select a pilot CMS. In comparison. University #1 had more 
opportunities to utilize an REP for their evaluation and selection process, and since 
each participant was already familiar with the process. University #1 was able to 
include additional information that further represented their needs. University #2 did 
not have this opportunity, but since it was an on-going evaluation period, the 
institution was able to ask questions and add information as needed. 

University #1 participant roles differed in the evaluation and selection process 
from that of University #2. The participants of University #1 were all part of a 
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committee made up of both faculty and staff members, with a specific task to evaluate 
and select a CMS. Each participant had equal input into the process, and was 
encouraged to express their concerns. Each participant came to the committee with a 
particular interest in using a CMS. Eor example, Jim, a department chair, utilized 
Netscape to create web pages. He then migrated to TopClass in order to utilize a 
system that would better manage his course materials. Nancy, also a department chair, 
consistently used technology in her courses and identified the use of a CMS as a way 
to challenge her to become a better teacher. Both decided to participate on the 
committee so that they could have input in choosing the institution’s CMS. All staff 
participants of University #1 were employed as support personnel and were equally 
responsible for selecting the right system that would serve the overall needs of the 
faculty and institution, including pedagogical support, technical support, and budget. 

University #2 participants were either faculty members or technology support 
staff and their roles varied. The faculty participants’ main role was to evaluate the 
system and at the same time develop pilot courses. Eor example, Dan, a faculty 
member in Biology, was interested in utilizing a CMS to distribute content on-line and 
to have a place for students to post daily discussions. Judy, a faculty member, 
combined her time in the Music and Technology Departments. Dan and Judy both 
participated in the evaluation and selection process because of their interest to 
incorporate specific content for discussion and evaluation on the web utilizing a CMS. 

The roles of the staff members varied and all did not participate concurrently. 
Eor example, Cynthia entered the process after the system was chosen and was 




71 



charged with creating strategies to enhance the adoption and diffusion processes, 
while others, such as Ron, participated in the beginning and were able to have input 
into the selection process. David and Judy, however, participated during the pilot 
selection phase and actually selected the CMS. In addition, Judy, a staff member, and 
David, an administrator, were also faculty members, thus, their roles featured both 
evaluation of the systems as a faculty member and selection strategies as end-users. 

University #1 participants viewed this process as time consuming, yet easier 
the second time because participants all felt they were eomfortable working with one 
another and thought it was easier to work with the same group over a long period of 
time. Having participated in several evaluation and selection processes. University #1 
had many chances to revise the process. In eontrast. University #2 approached the 
process quite the opposite, incorporating the entire experience as an evaluation and 
selection phase over a period of one year, and viewed the experience as an on-going 
process. 

In reference to concerns, both institutions expressed issues with costs and 
support. University #1 had more time to address these issues with vendors. University 
#2 regrettably had to evaluate long-range costs and support issues over a period of 
only one year and include these issues in their decision-making process. Overall, both 
institutions chose Blaekboard because of its entry level costs and features, and also its 



stability in the CMS market. 
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Adoption and Diffusion Processes 

University #1 had three opportunities to experience the adoption and diffusion 
process whereas University #2 had only one. Each time, University #1 increased the 
number of faculty using the CMS. For example, the first adopted CMS, TopClass, was 
introduced through early adopters and featured no workshops. The second and third 
CMSs (Prometheus and Blackboard) included incentives for faculty to attend 
workshops, which increased the number of users over time. University #2 also 
conducted workshops to entice their faculty to adopt Blackboard. This included 
inviting guest speakers from other liberal arts institutions to showcase what they were 
doing in their courses and offering the same opportunity to their faculty. Providing 
these workshops as incentives allowed both institutions to increase the number of 
users in their own ways. 

Participants at University #1 also experienced the adoption process with mixed 
emotions. Jim, a department chair, approached the Prometheus adoption with caution 
because he experienced problems with the server going down. Nancy, also a 
department chair, experienced the opposite, as she engaged herself in the role of a 
trainer. Bob, a systems administrator, felt the adoption and diffusion process of 
Prometheus was a relief because he did not have to manage TopClass, the first adopted 
CMS, and Prometheus at the same time. Al, the head of the Technology Department, 
considered the adoption and diffusion process of both Prometheus and Blackboard as a 
success because the committee learned from their experiences with selecting 
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Prometheus, and later the university added incentives for faculty to participate in 
workshops, thus supporting adoption. 

University #2 participants experienced the adoption and diffusion process in a 
unique manner. For example, Dan, a Biology faculty member, recalled the issue he 
had when adopting Blackboard was that many times Word documents would not open 
because the program was not loaded on all computers to which the students had 
access. Dan felt it was difficult to use the system if the students could not access the 
files. Judy recalled issues of support from Blackboard slowed down the diffusion 
process. David, head of the Technology Department, incorporated strategies such as 
think tank sessions for faculty to talk about ways to use Blackboard, which helped 
others to consider adopting the CMS. Accordingly, the entire process was incorporated 
within one academic year in which participants viewed the adoption and diffusion 
process as a yearlong unique experience. 

Although each institution experienced the adoption and diffusion process 
differently, both seemed to express issues concerning the use of their selected CMS. 
Since University #1 had more experience with the process, it became easier to create 
strategies over time. However, the approach of University #2 allowed for more 
interaction in less time and demanded a quicker adoption and diffusion process. 
Overall, each institution benefited from their experiences and input from faculty and 



staff. 
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Summary 

This chapter described the experiences of participants while being engaged in 
the process of evaluation, selection, adoption and diffusion of the CMS chosen by 
their institution. Both cases presented two distinct versions of the process, with 
similarities and contrasts found in each case. Based on these findings, the diffusion of 
innovation theory is offered in Chapter 5 as a theoretical explanation of the 
experiences of these participants. Further recommendations and suggestions are also 
offered for future studies of this nature. 




CHAPTER 5 



DISSCUSSION AND RECOMMENDATIONS 

Chapter 5 includes a discussion of Rogers’s diffusion of innovation theory as a 
framework for this study. The theory is applied to explain the processes of evaluation, 
selection, adoption and diffusion discussed in this study. In addition, this chapter also 
suggests potential recommendations for future studies. 

Introduction 

In recent years, distance education technologies such as course management 
systems have inspired opinion leaders to advise decision makers of the advantages and 
disadvantages in their use. The findings of this study indicate opinion leaders 
communicated the need for change, provided information and advice, and influenced 
the opinions of others associated with University #1 and University #2 to evaluate, 
select, adopt and diffuse a CMS. 

The following is a discussion of the diffusion of innovation theory, which 
provided the theoretical implications of this study as a framework to analyze the 
process of evaluation, selection, adoption, and diffusion of a CMS. 
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Discussion of the Diffusion of Innovation Theory 

As defined earlier, diffusion is “the proeess by which an innovation is 
communicated through certain channels over time among the members of a social 
system” (Rogers, 2003, p.ll). The four key elements of diffusion are: (1) the 
innovation, (2) eommunieation channels, (3) time, and (4) the soeial system. The four 
elements help describe the rate of time by which the innovation is adopted. These four 
elements served to help explain the proeesses of evaluation and seleetion of CMSs 
observed in this study. 



Evaluation and Selection 



The first element, an innovation, does not have to be new but is pereeived by 
an individual as being new (Rogers, 2003, p. 12). In this study, the innovation is a 
CMS. Members of both institutions gained knowledge of eourse management systems 
while attending a eonference. Naney, a department chair in Education of University 
#1, introduced others to the concept: 



I came back from the conference and explored what I was reading 
about course management systems, and I did not connect at that time 
with the College of Edueation faculty concerning the concept of a 
CMS. So I looked for people who were also interested in using a CMS. 



Ron, a former employee of the Technology Department of University #2, was 
also introduced to the coneept at a conference and began making others aware of the 



concept: 
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My experience opened up to researching CMS by seeing what other 
schools were doing. I started to correspond with liberal arts colleges in 
central Indiana and basically introduced myself and asked what they 
were using. 

The purpose of introducing the innovation is to stimulate a response and 
identify favorable or unfavorable attitudes towards the innovation. Rogers (1995) 
distinguishes two types of information with respect to innovations that may stimulate a 
response: (1) software information and (2) innovation evaluation information (Rogers, 
1995, p. 14). The first, software information, stimulates responses such as “What is the 
innovation?” and “How does it work?” and reduces uncertainty about the cause-effect 
of achieving the outcome. The second, innovation evaluation information, concerns 
“What are the consequences, advantages and disadvantages?” Both institutions 
expressed concerns during their evaluation and selection processes that reflect 
software and innovation evaluation information issues. University #1 developed an 
RFP (Appendix E) that emphasized concerns about the innovation (Prometheus and 
Blackboard) and how it works, in order to decrease uncertainties of choosing the 
CMS. University #2 also expressed concerns throughout their evaluation period and 
conducted workshops to emphasize the advantages of using Blackboard. Ultimately, 
each institution chose their CMS upon feeling comfortable to “incorporate the new 
innovation into their existing practices” (Rogers, 2003, p. 168). 

The second element, communication channels, involves the means by which 
the message is conveyed to an individual or individuals in order to reach a decision to 
adopt an innovation. This process is usually comprised of four essential elements: (1) 
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an innovation, (2) an individual or unit that has utilized the innovation, (3) another 
individual or unit without knowledge of the innovation, and (4) a means of 
communieating and linking the innovation to both units (Rogers, 2003, p. 18). During 
the evaluation processes, both University #1 and University #2 utilized 
communication channels to involve individuals in the evaluation and selection of their 
CMS. These participants were made aware of the issues concerning the CMS through 
interpersonal (face-to-face) communication. Rogers (2003) states: 

Most people depend mainly on subjective evaluation of an innovation 
that is conveyed to them from other individuals who have already 
adopted the innovation. 

Hence, each institution relied upon communication from individuals who were 
familiar with using the CMS to inform and persuade others to evaluate and adopt the 
system. 

The third element, time, follows a linear progression from the introduction of 
an innovation through its adoption or rejection. The time dimension is incorporated in 
diffusion through the innovation decision process, in which an individual passes from 
knowledge of an innovation, to adoption or rejection. This includes a measurement of 
the relative earliness or lateness when an individual adopts an innovation and 
observation of an innovation’s rate of adoption in a system (Rogers, 2003, p. 20). 

Time is one of the most important elements of the decision-making process. In this 
study, many individuals were involved and it took a considerable amount of time to 
evaluate their input and decide whether to adopt the CMS. 
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The fourth element, social systems, is “interrelated units that are engaged in 
joint problem solving to accomplish a common goal” (Rogers, 2003, p. 23). 

The structure of each system determines the type of communication patterns. For 
example. University #1 consisted of a committee to select their CMS. The patterns of 
communication were direct interpersonal channels such as committee meetings and e- 
mails, allowing decision-making processes to be more consistent. On the other hand. 
University #2 did not develop a committee; to a certain extent, they relied upon 
individuals who provided information as needed. Thus, the structure of each system 
could facilitate or impede the process of diffusion (Rogers, 1995, p.25). In this study, 
the structures of University #1 and University #2 and the processes used to 
communicate did not impede their decision making. The communication processes 
did, however, emphasize communication structures, that facilitated each institution’s 
process of diffusion (Rogers, 2003, p.24). 



Adoption and Diffusion Processes 



To further explain the process of adoption and diffusion, this study utilized 
Rogers’s (2003) innovation decision process. The following is a theoretical discussion 
of the innovation decision process and its relationship to this study. 
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The Innovation Decision Process 

Rogers (2003) states that the innovation decision process is used to gather 
information about an innovation and reduce uncertainties about its advantages and 
disadvantages (p. 172). The innovation decision process consists of five main steps: 
knowledge, persuasion, decision, implementation, and confirmation. 

Knowledge 

Rogers (2003) proposes that the innovation decision process begins with 
knowledge of an innovation when an individual or decision-making unit becomes 
aware of an innovation or idea and begins to understand its functions and use (p. 171). 
This stage includes certain behavioral patterns by an individual that may influence the 
communication messages in which the innovation is introduced. In the case of 
University #1, Nancy, a department chair in Education, discussed her knowledge of 
course management systems with other members of the faculty. That communication 
helped to facilitate the knowledge and awareness of the technology to other faculty 
members. Likewise, Ron, a former Technology Department staff member of 
University #2, also communicated the knowledge of course management systems to 
other members of the institution, which also resulted in awareness of the technological 
innovation. Thus, members of each institution communicated knowledge of the 
innovation, which influenced the way in which the innovation was introduced within 



their social system. 
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Persuasion 



The next phase in the innovation decision process is persuasion, which allows 
an individual to exhibit an attitude (favorable or unfavorable) about the innovation. 
This stage also incorporates forward planning in which individuals look at the 
advantages and disadvantages of how the innovation would affect their use (Rogers, 
2003, p. 174). 

Rogers (2003) states: 



All innovations carry a degree of uncertainty for an individual who is 
typically unsure of the new idea’s functioning and thus seeks social 
reinforcement from others of his or her attitude towards the innovation. 



(p. 175) 



Al, the head of the Technology Department of University #1, describes how 
individuals were persuaded to use Blackboard: 



The major thing that helped adoption was a mass of people using it, and 
students asking other instructors why don't they use this. Also, one 
faculty would mentor the other, to help it spread faster. 



Cathy, a member of the Technology Department of University #2, described 
persuading faculty to use Blackboard: 



Letting the faculty know that Blackboard was available without being 
pushy was a concern. We had a lot of training sessions for technical 
ways to use Blackboard and a lot of one-on-one consultation with 
faculty. We also had other discussions for concerns and suggestions, 
and we conducted two week-long workshops in January and May. 
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Both institutions used their own strategies to persuade faculty to use 
Blackboard. University #1 faculty mentored other faculty members in order to 
persuade them to use Blackboard. On the other hand. University #2 did not want to 
push the technology on faculty; rather, they offered training sessions and one-on-one 
consultation to persuade faculty members to use Blackboard. In the end, each 
institution was able to persuade faculty members to adopt Blackboard. 

Decision 

A decision is made in the innovation decision process when decision makers 
“engage in activities that lead to a choice to adopt or reject the innovation” (Rogers, 
2003, p. 177). Also, a decision can be made to adopt an innovation that could later be 
discontinued and replaced by another innovation. University #1 announced its decision 
to adopt Prometheus through an e-mail correspondence (Appendix D), but it was later 
discontinued because the company sold it and replaced Prometheus with Blackboard. 
Al, the head of the Technology Department of University #1, described the decision to 
replace Prometheus and adopt Blackboard: 

When Blackboard took over, people were not happy. They liked 
Prometheus, and you knew Blackboard was going to drop Prometheus. 
Blackboard also looked like they were not going to make money on the 
five thousand dollar package, so they were going to try to migrate 
people to the thirty thousand [dollar] package. Unfortunately for us, we 
had very little choice. 

University #2 experienced different circumstances while making their decision 
to adopt Blackboard. Cynthia, a staff member in the Technology Department, 
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provided a comparison of making the decision to adopt Blackboard as a course 
management tool for the distance education format, as opposed to larger schools with 
distance education programs in place: 

A lot of bigger schools have a distance education program, whereas we 
do not. We provided a classroom environment with Blackboard to 
support the distance education format as opposed to a distance 
education structure. It is two different angles. 

Accordingly, each institution reached their decision to adopt Blackboard 
differently. University #1 adopted Blackboard to replace Prometheus because of its 
discontinuance. On the other hand. University #2 adopted Blackboard to support the 
distance education format instead of developing a full distance education structure. 
Thus, University #1 made their decision to adopt Blackboard because they needed to 
replace their existing CMS, whereas University #2 decided to offer a distance 
education format, and adopted Blackboard as their CMS tool. 

Implementation 

Implementation usually follows knowledge, persuasion, and decisions and 
occurs when an innovation has been put to use. Consequences and uncertainty 
generally persist in this phase. For example. University #1 adopted and implemented 
Prometheus but later choose to adopt Blackboard. University #2 adopted Blackboard, 
and later expressed their concerns when Blackboard raised their costs to acquire 



additional features. 
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David commented on the university’s decision to continue using Blackboard: 

The cost did rise, but there were good arguments to stay with Blackboard. 

You have to look at the trade-offs of time invested by faculty to develop their 
courses. So the question was: Do you abandon the current system for a new 
one, and invest more into transporting their eourses into a new system? 

As a result. University #2 decided to keep Blackboard, regardless of costs. 
Rogers (2003) adds that implementation peaks when “a point is reached in which the 
new innovation is institutionalized as a regular part of the adopter’s operations” and is 
considered the end of the implementation phase (p.l80). However, a form of re- 
invention may take plaee that allows the innovation to be reeonfigured to adapt to 
behavioral changes and use of the innovation. Hence, implementation of an innovation 
is an important step of the innovation decision process, which could lead to alteration 
or elimination of the innovation. In this study. University #1 discontinued the use of 
TopClass, and upgraded to Prometheus. And when Prometheus was discontinued. 
Blackboard was chosen as an upgrade. University #2, likewise, decided to eliminate 
the use of Discus as a class discussion tool, and replace it with Blackboard. Thus, 
when University #2 designated the discontinuance of Diseus as a elass discussion tool. 
Blackboard introduced a newer use of the innovation as a CMS. 

Confirmation 

In this stage, individuals seek reinforeements for using the innovation. This 
stage includes: recognition of the benefits of using the innovation, integrating the 
innovation into a daily routine, and promoting its use to others (Rogers 2003, p. 199). 
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The administration of University #1 offered stipends as incentives to attend 
workshops, enticing faculty to recognize the benefits of using Blackboard. These 
incentives also helped to promote and encourage others to incorporate Blackboard in 
their daily use. University #2 also featured workshops to highlight the benefits of 
using Blackboard. In addition, the Technology Department worked closely with 
faculty to identify ways to incorporate Blackboard into their daily routines. Overall, 
both institutions confirmed their decisions to adopt the CMS, and exhibited no 
conflicting messages. 



Summary of the Innovation Decision Process 



Each University engaged in the innovation decision process differently. For 
instance. University #1 participants were comprised of a committee to evaluate and 
select each CMS. Thus, their decision was a collaborative effort. Each participant 
became knowledgeable about Prometheus and Blackboard through other committee 
members, persuaded members of the institution to select these systems, made a 
collaborative decision to adopt these systems, and implemented plans to diffuse these 
systems. 

Participants of University #2, however, engaged in the innovation decision 
process as individual members of the Technology Department, and members of the 
faculty. Each participant was accountable for communicating the knowledge of 
Blackboard to others as well as the advantages of using the innovation. In both cases, 
individuals from each institution possessed knowledge of course management 
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systems, and passed this information on to others, who had little or no knowledge of 
the innovation. 

In contrast, communication channels within each institution followed different 
paths. University #1 had a small network of technology users within their social 
system, particularly the early adopters, who used Prometheus. Others began using 
Prometheus as well, when knowledge of the system passed through interpersonal 
communication channels such as e-mail and personal interaction. Also, the institution 
noticed a rise in users, when more faculty members began to adopt Blackboard as 
knowledge of that system began to spread. 

In contrast. University #2 provided an introduction to course management 
systems through their Technology Department. Workshops and demonstrations were 
utilized as interpersonal communication channels. 

In both cases, all participants held similar beliefs and shared their experiences 
with others. Also, persuasion occurred within both institutions, when participants 
attended workshops to observe how other faculty members were incorporating course 
management systems in their courses. These workshops ultimately led to others 
adopting these systems. After the selection and adoption of these systems, both 
institutions engaged in the diffusion process, which led to implementation and 
confirmation that the institutions would keep Blackboard as the adopted CMS. 
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Research Question 

This study was designed to answer the central question: What are the factors 
that guide the decisions in selection and use of a CMS within two small liberal arts 
institutions? Each institution will be discussed separately to summarize the factors that 
guided their decision-making processes. 

University #1 

The first CMS, TopClass, was purchased through a grant, received by a faculty 
member. A short time later, other faculty members began generating interest in 
acquiring another system, which led to adopting Prometheus. Blackboard later 
replaced Prometheus, but during each process, a committee of faculty, staff, and 
administration, consistently contributed their input. 

One of the factors guiding their decision to adopt Prometheus was the cost to 
purchase and maintain the system. Other factors included whether the system could 
handle pedagogical tasks such as importing files. The committee also considered 
features each system provided, such as grade book functions and a discussion tool. 
When Blackboard was selected to replace Prometheus, in addition to the 
aforementioned, the committee expressed concerns about the ability to transfer their 
existing courses from Prometheus to Blackboard. Therefore, support became another 
deciding factor in the selection of Blackboard. Al, the head of the Technology 
Department of University #1, explained how these factors were considered: 
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Blackboard offered some things that come with the system, and some you 
had to add on. Some things Blackboard offered have been more successful 
than others, like grade book and chat features, but that was a small part of 
the decision. 



Universit\ 



University #2 engaged in their evaluation and selection process over a period 
of one year. Some of the factors guiding their decision were pedagogical features and 
support, while cost was not a concern. The input of faculty also helped to guide the 
institution’s decision. Many were concerned with purchasing a system that would 
work with all disciplines on campus as well as purchasing a system that was easy to 
use. David, the head of the Technology Department of University #2 adds, “They 
wanted something simple to use, similar to a word processor, and easy to transfer 
materials on-line.” Overall, the institution settled on Blackboard because it seemed to 
fit pedagogically with faculty needs. 



Implications of this Study 

To gain an understanding of the processes of evaluation, selection, adoption 
and diffusion of a CMS, this study focused on Rogers’s (2003) diffusion of innovation 
theory. The results of this study supports Rogers’s innovation decision process, in 
which knowledge of an innovation, if communicated over a period of time within a 
social system, can influence the outcome of the adoption of that innovation. The 
innovation decision process explains how these stages relate to the evaluation, 
selection, adoption and diffusion processes discussed in this study. For example. 
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knowledge of an innovation oecurs during the evaluation proeess in which adopters 
becomes aware of the needs of the innovation and evaluate the innovation accordingly, 
Persuasion occurs when individuals form favorable or unfavorable attitudes towards 
the innovation, and lean towards the selection of the innovation. Decision reflects the 
individual’s choice to adopt the innovation. Implementation occurs when the adopter 
puts the innovation to use, and confirmation seeks to reinforce their decision and 
engage in the diffusion process (Figure 7). 



Knowledge 


► Evaluation 


Persuasion 


►Selection 


Decision 


— ► Adoption 


Implementation/Confirmation 


► Adoption/Diffusion 



Figure 7 . Five steps in the innovation decision process 

This study also suggests that the processes used to evaluate and select a system 
may affect the outcome of adoption. In both cases, each institution engaged in the 
process at their own pace, and used different resources to achieve their goals to select 
and adopt a system. For instance. University #1 developed an RFP to help guide their 
evaluation and selection processes, and used a committee to collect input. University 
#2, on the other hand, provided access to Blackboard on another institution’s system, 
which allowed participants an opportunity to evaluate Blackboard and provide 
feedback during the evaluation process. 
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Accordingly, the stakeholders involved played an important role in the evaluation and 
selection processes and ultimately influenced the outcome of adoption and diffusion. 

Recommendations Related to Study 
This study included the experiences of administrative, staff, and faculty 
participants. However, one limitation of this study was that it did not include the 
impact of future technological innovations on course management systems. 

Course management systems are eontinuing to evolve. Their uses have far 
outgrown placing documents on the web for retrieval, into advanced utilizations of e- 
leaming applications. While engaging in the evaluation and selection processes of 
course management systems, institutions should eonsider current and future trends in 
technology. Cynthia, a staff member in the Technology Department of University #2, 
proposed: 

The students coming in see technology as not something to learn, it is 
something they have grown up with, and part of their daily routines, 
like i-pods and laptops, so it’s something we need to be aware of, 
especially the expectations of the students. 

Current ways to utilize a CMS, such as posting blogs and podcasts, are some of 
the newer pedagogical practices and trends in education. Thus, with newer innovations 
and new-found pedagogical uses of CMSs, vendors like Blackboard would have to 
build on existing products to adapt to current and future technological environments. 
And likewise, institutions would have to develop strategies to evaluate and select 
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systems, in accordance to current and future technological innovations and 
pedagogical practices. 



Recommendations to Institutions 



E-learning technologies have flourished over the past decade. And as 
information technology departments continue to plan, evaluate, select, and adopt 
CMSs, many will debate whether to develop their own systems, choose a vendor- 
based system, or acquire an open-source system. Open-source systems are course 
management systems offered with a free to use license agreement, developed by 
individual users or a community of users (Waterhouse, 2005, p.7). Participants in a 
2007 campus computing survey indicated: 



Almost three-fifths (57.3 percent, compared to 51.9 percent in 2004) 
agree that "Open-source will play an increasingly important role in our 
campus IT strategy." However, less than a third of the survey 
respondents (27.6 percent, compared to 28.9 percent in 2004) agree that 
Open-source "offers a viable alternative" for key campus administrative 
or ERP applications such as student information systems, campus 
finance systems, or personnel/human resource software. (Green, 2006, 
P-3) 



These statistics state that open-source systems continue to grow among college 
campuses. Open-source systems offer several advantages and disadvantages. One 
advantage of open-source systems, in comparison to a vendor-based system such as 
Blackboard, is that there is no cost to purchase the software. However, the debate over 
support persists. Since open-source systems are usually supported and maintained by a 
community of users, institutions would have to consider the long-range costs of 
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maintaining systems (i.e., staffing, system maintenance and upkeep, and server 
specifications). Institutions adopting open-source systems would need resources that 
would allow funds to be allocated for staffing and system maintenance. Bob, a staff 
member in the Technology Department of University #1, pointed out, “From a 
financial standpoint, you save from fees, but the staff has to be bigger.” 

Institutions with the financial resources to adopt an open-source system should 
consider three options. First, the institution must analyze their financial commitment 
to maintain an open-source system. This includes hiring a system administrator, 
technical personnel to maintain the system’s infrastructure, training staff, and 
providing a fully functional help desk. This also includes implementing, at the least, a 
five-year plan to allocate funding as needed. 

Second, institutions should examine how they plan to use an open-source 
system. For example, Lynn University in Boca Raton, Florida, with an enrollment of 
2,400 students, adopted uPortal as an open-source system to help coordinate all 
student-oriented systems (Villano, 2006, p.l). 

What is not known, however, is whether Lynn University has a similar 
administrative, faculty, and staff structure as University #1 or University #2. And 
furthermore, research and statistics are lacking to indicate the costs Lynn University 
absorbed and benefits, if any, of adopting the uPortal system. Hence, reasons for 
adopting an open-source system, and resources allocated to do so may be available for 
one institution but may not fit as an option for other smaller institutions. 
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Next, when deciding to adopt a new system, institutions should conduct a 
survey, such as the one University #2 designed (Appendix F), to assess their on-going 
faculty and institution wide needs for a CMS. This survey could be used to help the 
institution evaluate their current system, and decide whether to discontinue or continue 
its use, develop their own system, or adopt an open-source system. In addition, an RFP 
should be created, such as the one University #1 developed (Appendix E), to evaluate 
vendors’ responses to the RFP in accordance to the institution’s needs. Unfortunately, 
creating an RFP would not be an option if the institution decided to adopt an open- 
source system because open-source systems are not bid on like vendor-based systems. 
Alternatively, if an institution does not have the resources to adopt an open-source 
system, they should consider the adoption of a vendor-based system, such as 
Blackboard. And although costs would be a factor, the institution would have access to 
the vendor for answers to issues related to costs and maintenance. 

Recommendations for Future Research 

While institutions continue to evaluate e-learning pedagogical needs, 
additional literature is needed to offer an approach and solutions to help them evaluate 
and select a CMS. Since this study utilized two small liberal arts institutions, other 
institutions would find this study useful, particularly if their administrative structures 
appear parallel or if their needs fit the descriptions within this study. In particular, 
small liberal arts institutions will benefit from further research that highlights plans to 



evaluate, select, and adopt a CMS. 
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Overall, there is a lack of qualitative studies emphasizing the processes of 
evaluation and selection of a CMS. A qualitative study such as this would add to the 
literature concerning the adoption and diffusion of innovations and would be valuable 
to decision makers planning to adopt a CMS and other educational technologies. 
Qualitative research is also needed to assess the benefits of small liberal arts 
institutions adopting open-source systems. And further qualitative studies could also 
identify how small liberal arts institutions are implementing and using other 
educational technologies. Furthermore, this study could be designed as a quantitative 
study to identify how many liberal arts institutions have planned or are planning to 
evaluate and select a CMS and how these systems are being used. 

Additionally, postevaluation studies should be done to assess the following 
considerations. First, a study to assess the real costs of operating a system after it’s 
been adopted, would benefit institutions planning their operating expenses over a 
period of years. Next, a study indicating the cost of abandoning a system would help 
institutions understand the costs, if any, to discontinue use of their CMS. Research is 
also needed to reveal costs to support the evaluation of systems, and if there would be 
an increase in price for additional users. For example, if institutions adopt a system 
with the intention of serving 2,000 users and over a period of two years that number 
increases to 3,000, a study to address the expenses incurred, if any, to increase the 
number of users, would be helpful for institutions to plan for additional funding 



towards capital expenditures. 
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Studies are also lacking to indicate which systems students prefer to use, what 
systems are actually being used, and how these systems are being used by students. 
And finally, studies are needed to evaluate the costs of operating open-source systems 
compared to vendor-based systems like Blackboard. Decision makers would greatly 
benefit from these types of studies, particularly if they are weighing the advantages 
and disadvantages of adopting open-sources vs. vendor-based systems. 

Conclusions 

In conclusion, e-learning applications such as course management systems, 
whether vendor-based or open-source platforms, will continue to drive technological 
advancements in education. Inevitably, institutions will continue to rely on future 
developments of e-leaming applications, and as technologies change, institutions will 
be forced to adapt and provide practical and logical solutions. 

I believe in the next ten years open-source systems will become a norm in the 
CMS landscape. And while course management systems are currently the trend in 
distance education technology, the future represents enormous possibilities. 
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Date: Wed, 25 Jan 2006 13:05:57 -0600 
From: “Phillip Powell” <ppowell@valpo.edu> 

Message-Id: <1 138215957-456.00057.00312-smmsdV2.l.3@mailhub.valpo.edu> 

X-SMMS-Source: 152.228.140.135 

To: 

Subject: Research Study 
Hello, 

My name is Phillip Powell. I am a graduate student at Northern Illinois University. 

Fm currently looking for participants for a study which involves the process of 
evaluation, selection, adoption and diffusion of a course management system. I would 
like to speak with you more about this opportunity if you or individuals within your 
institution may fit the criteria of the study. Attached is a formal document for your 
review only. It is intended to give you some background information of the study only. 
I would greatly appreciate a response to this e-mail indicating a phone number and the 
best time to contact you. 

Thank You 
Phillip Powell 
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My name is Phillip Powell. I am a doetoral candidate at Northern Illinois University in 
the Department of Educational Technology Research and Assessment. I am contacting you in 
reference to your participation in a study to be used as data in completion of a dissertation for 
a doctoral degree in Instructional Technology. 

The purpose of this study is to examine the processes undertaken by institutions that selected a 
course management system and to provide insight into participant experiences during the 
selection process. The knowledge gained from this study may help to recommend future 
research related to evaluating and selecting technology, and also provide information 
particularly useful for small liberal arts institutions considering the selection of a CMS. 

You will be asked to respond to interview questions, containing items pertaining to the 
evaluation, selection, adoption and diffusion of your institutions course management system, 
which should take about one hour to complete. These questions will be audio recorded and 
kept in a locked safe to secure the identity of participants. There are no anticipated risks, 
however, if you feel uncomfortable during or after the interview process, the researcher will 
be available to talk with you and, if you wish, offer you a referral to a counseling agency. 

All of the information obtained from your participation will be kept confidential. Your consent 
form will be kept separate from the data and the data will not be available to anyone other than 
the researcher conducting the study. 

You are reminded that your participation is voluntary. This means that you can choose to stop 
at any time. If you have any questions or concerns related to your participation in this study, 
please contact Phillip Powell at 708-229-0409. Any questions about your rights as a research 
participant can be addressed to the NIU Office of Research Compliance (815-753-8588). 

Background on Study: 

In recent years, the value of course management systems have increased partly due to the 
ability to reach a wider audience. A 2002 survey conducted by tbe Campus Computing Project 
suggests at least one-fifth of college courses were utilizing CMS’s and three-quarters of higher 
education institutions have adopted a CMS, a rise from 14.7 percent in 2000 to 26.5 percent in 
2002 (Green, 2002, www.campuscomputing.net). Accordingly, the purpose of this study is to 
examine the processes undertaken by institutions that selected a Course Management System 
and to provide insight into participant experiences during the selection process. In order to 
gain a deeper understanding of the Course Management System selection process and 
experiences of stakeholders involved, the following central question will guide this research 
for data collection and analysis: What are the factors that guided the decisions in selection and 
use of a CMS within small liberal arts institution? The following subquestions will also be 
used to guide this study: 

1. What were the issues leading to the need for a CMS? 

2. What kinds of procedures were used in the evaluation process? 

3. What were the roles of the administration, staff, and faculty in the evaluation and 
selection processes? 

4. How was the CMS selected by the organization? 

5. How is the CMS being used in the organization? 
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I agree to participate in the research project titled Case Study: The evaluation, selection, and 
adoption of a course management system, being conducted by Phillip Powell, a graduate 
student at Northern Illinois University. I have been informed that the purpose of the study is to 
examine the processes undertaken by institutions that selected a course management system 
and to provide insight into participant experiences during the selection process. The estimated 
completion of this study is December of 2006. 

I understand that if I agree to participate in this study, I will be asked to do the following: 
respond to interview questions related to your experience as described in the purpose of this 
study. The interview will be audio recorded and the process will take about one hour to 
complete. 

I am aware that my participation is voluntary and may be withdrawn at any time without 
penalty or prejudice, and that if I have any additional questions concerning this study, I may 
contact Phillip Powell at 708-229-0409. I understand that if I wish further information 
regarding my rights as a research subject, I may contact the Office of Research Compliance at 
Northern Illinois University at (815) 753-8588. 

I understand that the intended benefits of this study include providing stakeholders with 
information useful in planning for the evaluation and selection of a course management 
system. 

I understand that all information gathered during this study will be kept confidential by 
referring to the participants generically as a Faculty, Staff, or Administrative member of the 
named institution, or given pseudonyms. In addition, all data collected from this study will be 
kept confidential and the researcher will be the only person with access to this information. 

I realize that Northern Illinois policy does not provide for compensation, nor does the 
University carry insurance to cover injury or illness incurred as a result of participation in 
University sponsored research projects. I understand that my consent to participate in this 
project does not constitute a waiver of any legal rights or redress I might have as a result of 
my participation, and I acknowledge that I have received a copy of this consent form. 



"I have read the information about the study and have been informed of it's general purpose. I 
am fully aware of the risks and benefits associated with participating in the study described to 
me. I acknowledge that I have received a copy of the informed consent form and agree to 
participate in the study. I understand that I can withdraw at any time without penalty." 



Signature of Subject/Date 
Permission to Record Audio 



Print Name 



Today’s Date 



Signature 
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Thu, 15 Feb 2001 10:33:31 
From: 

Subject: Prometheus Courseware 
To: Faculty 

Colleagues: 

Many of you have been waiting for this. The University has committed itself to a new 
courseware package that, according to the task force, will change your life — at least 
transform your teaching! The announcement comes from XXX XXX, whom chaired 
the group. I am very grateful to our colleagues on this task force for their hard work. 
So READ on. 

****Prometheus Courseware Selected for University #1**** 

Last summer, the Provost's Office established a task force to identify, evaluate, and 
recommend a courseware product for use in support of placing course materials online 
at University #l.The task force issued an RFP to 22 potential vendors. This week the 
task force made a recommendation to purchase Prometheus software for University #1 
and the recommendation is being implemented. Prometheus is a database driven, web- 
based courseware application. Placing your course materials online is simple through 
its user-friendly, step-by-step electronic forms. You can customize the design to have 
the feel and look you want. You can send and track e-mail and class announcements, 
allow students to have round table type discussions on their own time schedule by 
posting messages, and create separate electronic bulletin boards for groups to discuss 
projects among themselves. You can communicate through class chat rooms, create 
and administer tests with multiple choice, true/false, short- and long-answer questions, 
have the computer generate test questions randomly for students taking the test side by 
side, let Prometheus grade your true/false and multiple choice answers, and put the 
results in your electronic grade book. These are just a few of the many features that 
will be at your disposal with Prometheus. For more information go to 
http://www.prometheus.com/ . The staff is now working with Prometheus staff to get 
the courseware installed as soon as possible. Look for an announcement soon about 
open demonstrations of Prometheus, which will then be followed up with ongoing 
hands-on workshops so that you can put your course online. If you bring your course 
content to the workshops, your course will be online before you leave — we guarantee 
it! Prometheus is an outstanding product that will help to enhance teaching and 
learning at University #l.June 27, 2000 
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UNIVERSITY #1 COVER LETTER AND REQUEST EOR 
PROPOSALS EOR WEB-BASED COURSE SUPPORT SOETWARE 
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<name> 

<address> 



Dear <name> 

We are pleased to offer the Request for Proposals from University #1 for a Courseware 
Support system for your consideration. We hope you will be interested in submitting a 
proposal for a system to enhance the use of instructional technology at University #1. 

Enclosed is a paper copy of the RFP, and a Windows/Intel diskette containing the RFP in both 
Microsoft Word 6.0 and ASCII text format. The diskette also contains an Microsoft Excel 5.0 
worksheet to aid you in preparing your response. We encourage you to use these resources. 
The README.TXT file on the diskette describes the contents of the diskette more fully. 

We expect Vendors to incorporate this information in their responses. We assume Vendors 
will identify themselves fully in their responses; we have not included a place for Vendor 
identification on these forms. 

In cases where Vendors do not feel comfortable quoting costs beyond their control (such as 
staff salaries), they must provide sufficient information for the University to calculate those 
costs. 

Friday, July 7, 2000 is the deadline for questions if you wish to be sure of a response. 
Questions should be submitted in plain ASCII text via electronic mail with subject line 
containing RFP. E-mail attachments in proprietary format (e.g., Microsoft Word or 
WordPerfect) are not acceptable. Even if you have no questions, you should send a message if 
you want a copy of the questions and answers. Questions will be consolidated and vendor 
identification removed (unless the question itself reveals that information) and all vendors will 
receive a response by e-mail by Friday, July 14, 2000. Questions received after the deadline 
will be answered by e-mail Reply; all vendors on the mailing list will receive copies of the 
reply. 

Friday, August 4, 2000, at 2:00 PM is the deadline for proposals. Proposals are to be 
delivered to the office of the Vice President for Administration and Finance of University 
#1. Vendors are solely responsible for ensuring that their proposals are delivered on or before 
this deadline. 

You will find the complete anticipated schedule in Section 9.15 of the RFP. 

We look forward to your response to this Request for Proposals. 

Sincerely, 

Executive Director, Electronic Information Services and 
Campus Contact for the Course Support System 
University #1 
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November 11, 2002 

We are pleased to offer the Request for Proposals for a Courseware Support system for 
your consideration. We hope you will be interested in submitting a proposal for a system 
to enhance the use of instructional technology. 

Enclosed is a paper copy of the RFP, and a Windows/Intel diskette containing the RFP in 
both Microsoft Word 97 and ASCII text/HTML format. The diskette also contains a 
Microsoft Excel 97 cost worksheet to aid you in preparing your response. We encourage 
you to use these resources. The README.TXT file on the diskette describes the contents 
of the diskette more fully. 

We expect Vendors to incorporate this information in their responses. We assume 
Vendors will identify themselves fully in their responses; we have not included a place for 
Vendor identification on these forms. 

In cases where Vendors do not feel comfortable quoting costs beyond their control (such as 
staff salaries), they must provide sufficient information for the University to calculate 
those costs. 

Monday, November 25, 2002 is the deadline for questions if you wish to be sure of a 
response. Questions should be submitted in plain ASCII text via electronic mail with 
subject line containing RFP. E-mail attachments in proprietary format (e.g., Microsoft 
Word or WordPerfect) are not acceptable. Even if you have no questions, you should send 
a message if you want a copy of the questions and answers. Questions will be 
consolidated and vendor identification removed (unless the question itself reveals that 
information), and all vendors will receive a response via e-mail by Friday, December 13, 
2002. Questions received after the deadline will be answered by e-mail reply; all vendors 
on the mailing list will receive copies of the reply. 

Monday, January 6, 2003, at 2:00 PM is the deadline for proposals. Proposals are to be 
delivered to tbe Office of the Vice President for Administration and Finance of University 
#1. Vendors are solely responsible for ensuring that their proposals are delivered on or 
before this deadline. 

You will find the complete anticipated schedule in Section 9.13 of the RFP. 

We look forward to your response to this Request for Proposals. 

Sincerely, 



Executive Director, Electronic Information Services and 
Campus Contact for the Course Support System RE 
University #1 
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Web-Based Course Support Software 
November 11, 2002 

1 . Overview of University #1 

University #1 is an independent, church-related, residential university with a 
full-time equivalent student population of about 3200. 

2. Objective 

University #1 is seeking a standards-based, integrated course support software 
system which will: 

■ Support courses in all major academic areas; 

■ Be flexible and easily modifiable; 

■ Provide a consistent user interface and excellent tools for instructors 
and students to interact with the system; 

■ Suit University #1 size, staff, and institutional culture. 

3. Current Status 

3.1 Course Support 

3.1.1 Course-support software based offerings 

3. 1.1.1 Approximately 100 faculty members use Prometheus. 

3.1.2 Free-form offerings 

A number of faculty currently use technology for instruction in 
a number of different ways. 

3 . 1 .2. 1 Every course/section has a list-server-based mailing 
list. 

3.1 .2.2 Electronic mail is routinely used for class 
communication in about 80% of classes. 

3. 1.2. 3 Web-based course materials such as syllabi and 
reading lists are used by approximately 40% of 
faculty. 
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3.2 Network 

University #1 is fully networked, with fiber optic backbone and 
lObaseT Ethernet in the buildings. Currently, most of the backbone 
consists of 100Mb or 1000Mb Ethernet; Wireless networking will be 
phased in over the next several years (in addition to the fiber and wire). 
Protocols used are TCP/IP for UNIX servers and both TCP/IP and IPX 
for Novell servers. 

3.3 Servers 

3.3.1 Electronic Mail is supported by an IMAP4 server running on a 
SunEire 3800 under Solaris 8 (SunOS 5.8). 

3.3.2 The Web is supported by Apache 1.3.26 running on a Sun 
Enterprise V880 server; web pages are submitted through a 
staging server, which is Apache 1.3.26 running on a Sun 
Enterprise 3000 server. Both servers are running Solaris 8 
(SunOS 5.8). 

3.3.4 The Library Online Public Access Catalog is Innovative 

Interfaces running on a Compaq Alpha AXP under DEC UNIX. 

3.3.6 Databases are supported by Oracle 8i (8.1.7) running on a Sun 
Enterprise 450 server. 

3.3.7 Authentication services are supported by Sun One LDAP 
Version 4.16 running on the same Sun Enterprise 3000 server as 
web staging. 

4. General System Requirements 
4.1 General Characteristics 

4.1.1 Advanced Technology: System design and platform must allow 
for recent and future developments in computer technology. 
Both system and platform should be proven but close to the 
beginning of the product life cycle. 
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4.1.2 Integrated: There should be one authoritative source for each 

data element, and all modules using a given data element should 
retrieve it from the authoritative source. In the event system 
design dictates multiple copies of a given data element, the 
system should present a single-system image, and updating the 
authoritative source should cause all copies to be updated 
immediately and transparently; and updating non- authoritative 
copies should not be possible. 

4.1.4 Open: System must permit easy import/export of information 
from/to a wide variety of other applications, including 
competing course support systems; desktop applications such as 
the Microsoft Office suite; World-Wide Web; the electronic 
mail system; the Datatel/Unidata administrative system; other 
DBMS-based systems, on and off campus; data warehouses; 
and executive information systems. System should permit 
direct access to information by other applications, and be 
capable of direct access to information stored in other 
applications. 

4.1.5 Modular: The design of the system must facilitate rapid and 
correct response to changing needs at University #1. 

4.1.6 Robust: the system should have sufficient intelligence to be able 
to warn the user when it receives corrupt or inconsistent data. 

4.1.7 Stable: The software must be technically stable and relatively 
bug-free. We expect prompt notification by the vendor of bugs 
reported by other users and receipt of appropriate patches or 
upgrades to address the problems. Explain how this will be 
accomplished. 

4.1.8 Frugal: the new system should simplify the maintenance of 
information to the point where individuals can organize and 
present their own information. This system should create new 
opportunities without encumbering the institution with 
specialized staff, maintenance overhead and rigid data and 
reporting structures. 
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4.1.9 Easy to Use: The system must have a consistent and lucid 
interface that facilitates use and simplifies complex activities; at 
the same time, sophisticated users should be able to apply their 
advanced knowledge. The supporting documentation (on-line 
and paper) must be accessible by nontechnical users and should 
facilitate the resolution of problems and the training of users. 
The vendor of the software will provide several levels of 
support including access to other users, timely telephone 
support, remote diagnostics of the system, initial on-site 
installation and training. 

4.1.10 Maintainable: the system will have an underlying structure that 
enables the vendor to improve its function, to respond to 
changes in hardware and software design concepts, and to 
minimize "bugs." Updates and "fixes" should require minimal 
downtime and minimal alteration of the existing structure. 

4.1.11 Secure: the system must ensure the confidentiality of key 
information. The system must provide various levels of 
security extending to individual data elements as well as 
security auditing of essential functions. 

4.1.12 Divisible: the system should permit the University to treat 
separate divisions as entities for certain purposes, while still 
avoiding duplication of data and data entry. For example, the 
undergraduate division, the School of Law, and Graduate 
Studies need to be treated as separate entities for certain 
purposes. 

4.1.13 Distributable: the system will function with full effectiveness in 
a heterogeneous LAN environment. 

4.1.16 Focused on locally developed courses as opposed to "canned" 
courses. 

4.3 Standards 

4.3.2 Interface should be provided to 

4.3.2. 1 Internet, including campus web site, intranets as 
appropriate, etc. 

4. 3. 2. 3 Commonly-used portal systems (list all those with 
which your product interfaces naturally) 
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4.3.3 Content format must allow for simple transfer of text, graphies, 
test questions, etc. to/from any of these formats that apply: 

4.3.3. 1 another vendor’s course support package (Prometheus, 
in particular) 

4. 3. 3. 2 plain text files 

4. 3. 3. 3 tab or comma-quote delimited flat files (for input to 
databases etc.) 

4. 3. 3.4 word processing files 

4. 3. 3. 5 databases, including test questions 

4. 3. 3. 6 presentation packages such as PowerPoint 

4. 3. 3. 7 other test banks 
4.3.3.S XML 

4.3.4 Content format will allow easy inclusion of the following file 
types 

4.3 .4. 1 standard hypertext (HTML/XML) 

4. 3.4. 2 standard ASCII text 

4.3.4.3 PDF 

4.3. 4.4 images (.jpg, .gif, . . .) 

4. 3.4. 5 Video (.mpg, . . .) 

4. 3.4. 6 Audio (.wav, .mp3, etc) 

4. 3. 4. 7 ZIP, GZ, BZIP, and other compressed file formats 

4.3.5 Open standard for communication with 

4.3.5. 1 University #1 Library resources (electronic reserves, 
databases, etc.) 



4. 3. 5. 2 Resources at other libraries 
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4. 3. 5. 3 Existing University database (Datatel) applieations for 
purposes such as importing class rosters of exporting 
grades 

5. System Functional Requirements 

The following subsections outline the requirements that are specific to the 
various system functions. 

5.1 System Functions: The system shall be capable of the following: 

5.1.1 Information Organization/Delivery 

5.1.111 Incorporate web pages for syllabus, assignments, 
readings, etc. 

5. 1 . 1 .2 Accommodate links to reference sites with ability to 
open new window at instructor's option 

5. 1 . 1 .3 Accommodate image database(s) 

5 . 1 . 1 .4 Automated glossary tool 

5 . 1 . 1 . 5 Automated index tool 

5 . 1 . 1 .6 Search tool for course content 

5 . 1 . 1 .7 Student presentations area 

5 . 1 . 1 . 8 Support for special characters (foreign language, 
mathematics, science, music, phonetic, etc.) 

5 . 1 . 1 .9 Capable, well-integrated equation editor 

5.1.1.10 Fectures section 

5.1.1.11 Proj ects section 

5.1.1.12 Course overview 

5.1.2 Communication 

5. 1 .2. 1 Bulletin Board 

5. 1.2. 2 Anonymous Bulletin Board 
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5.1.2.3 


Whiteboard 


5. 1.2.4 


Chat (synchronous communication) 


5.1.2.5 


Anonymous Chat (but identified and logged at system 
administrator level) 


5.1.2.6 


Logged Chat 


5. 1.2.7 


E-mail (via existing campus system) 


5. 1.2.8 


Threaded discussion list 


5.1.2.9 


V ideoconferencing 


5.1.2.10 


Voice chat 


5.4.2 Authorization 


5.4.2.1 


Authorization tools 


5.4.2.2 


Batch add instructors (from University database) 


5.4.2.S 


Batch add students (from University database) 


5.4.2.4 


Security access 


5.4.2.5 


Variable level of security 


5.4.2.6 


Guest account creation 


5.4.2.7 


Security features 


5.4.2.S 


Assignable administrator role 



6.5 Customer Support 

6.5.1 Help desk must be available, preferably on a 7x24 basis, for at 
least central technical staff, at no charge beyond normal annual 
maintenance charges. Describe this service and any limitations 
on who may call for help. 

6.5.2 If not included in 6.5.1, Help desk should be available, 
preferably also on a 7x24 basis, for instructors and students. 
Describe this service, its cost, and any limitations on who may 
call for help. 
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6.5.3 On-line problem-solving (at your site or at University #1) must 
be accessible on 7x24 basis. World-Wide Web interface is 
preferred. Describe this service. 

9.4 Proposal Submission 

9.4.3 Ten (10) copies of the Proposal response shall be separately 
packaged in a sealed envelope and identified with the name of 
Vendor and the designation, "Sealed Proposal for University #1 
Course Support System." These copies shall be delivered to the 
same place and at the same time as the original proposal. 

9.4.4. Two (2) copies of any supplemental material are requested. 
These copies shall be delivered to the same place and at the 
same time as the original proposal. 

9.13 Timetable 

Key Dates/ Anticipated Timetable 



RFP published and available Nov 11, 2002 

Vendors Send Questions Nov 25, 2002 

University #1 Responses to Questions Dec 13, 2002 

Proposal Delivery Date Jan 6, 2003 

Selection of Proposal and Award of Contract Mar 7, 2003 

System Delivery Apr 7, 2003 

Test Period Begins Apr 7, 2003 

Acceptance of Test Results May 7, 2003 

Production Begins May 14, 2003 

Acceptance of System May 14, 2003 



11. Costs 

The following cost tables are to be filled out and submitted with your proposal. 
A Microsoft Excel workbook is supplied on diskette for your convenience. 

It is assumed your system may be run on a shared server with no penalty in 
software license costs (including database). If this assumption is not correct, 
explanation and estimate of cost penalties are required. 
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FY03 FY04 FY05 FY06 FY07 Total 



INITIAL COSTS 



FY03 



System Initial Costs 

Initial License, Base (Itemize) 

Initial License, Per Site (Itemize) 
Initial License, Per Course (Itemize) 
Initial License, Per Capita (Itemize) 
Initial License, Modules (Itemize) 
Documentation, Base (Itemize) 
Documentation, Modules (Itemize) 
Training, Base (Itemize) 

Training, Modules (Itemize) 
Installation, Base 
Installation, Modules (Itemize) 
Conversion, Common (Itemize) 
Conversion, Modules (Itemize) 

FY03 



FY04 FY05 



FY04 FY05 



FY06 FY07 Total 



FY06 FY07 Total 



Server Capital Costs 
Hardware (Itemize) 

Initial License Fees 
Operating System 
Utilities (itemize) 

Installation 

Training 

Startup Supplies (if any) 

FY03 FY04 FY05 FY06 FY07 Total 



Network Costs 

Hubs, Routers, Bridges (Itemize) 

Internal building wiring (Itemize) 

Intra-building cabling (Itemize) 

Network Servers (Itemize) 

Network Software (Itemize) 

Other (Itemize) 

FY03 FY04 FY05 FY06 FY07 Total 



Workstation Costs 

New or Upgraded Workstation Hardware 
Software Licenses, Base Apps 
Software Licenses Related to Proposal 
Network Connections 
Other (Itemize) 
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FY03 FY04 FY05 FY06 FY07 Total 



Other Initial Costs (Itemize) 
TOTAL INITIAL COSTS 



OPERATING COSTS 
FY07 Total 



FY03 



System Operating Costs 

Annual Maint, Base (Itemize) 

Annual Maint, Per Site (Itemize) 
Annual Maint, Per Course (Itemize) 
Annual Maint, Per Capita (Itemize) 
Annual Maint, Modules (Itemize) 
Doeumntatn Updts, Base (Itemize) 
Doeumntatn Updts, Modules (Itemize) 
Continuing Trng, Base (Itemize) 
Continuing Trng, Modules (Itemize) 

FY03 



FY04 FY05 



FY04 FY05 



FY06 FY07 Total 



FY06 FY07 Total 



Server Operating Costs 

Hardware Maintenanee (Itemize) 

Software Maintenance 
Operating System 
Utilities (itemize) 

Continuing Training 
Supplies 

FY03 FY04 FY05 FY06 FY07 Total 



Network Operating (Reqd New Equipt) 

Hubs, Routers, Bridges (Itemize) 

Network Servers (Itemize) 

Network Software (Itemize) 

Other (Itemize) 

FY03 FY04 FY05 FY06 FY07 Total 



Workstation Operating Costs 
Software, Base Applications 
Software Related to Proposal 
Other (Itemize) 
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FY03 FY04 FY05 FY06 FY07 Total 



Staff Costs 

Staff Salaries (Itemize) 

Fringe Benefits 

FY03 FY04 FY05 FY06 FY07 Total 



Other Operating Costs (Itemize) 

FY03 FY04 FY05 FY06 FY07 Total 



TOTAL OPERATING COSTS 
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University #2 - Blackboard Survey - Spring 2006 




Blackboard Survey - Spring 2006 



Basic Survey (Page 2 of 4) 

This section will take approximately 5 minutes to complete. 



6 

In a given semester, in which of your classes do you use Blackboard? 



1 

None 


2 

Some 


3 

All 


For eReserves only? 






JJ 




-U 


For eReserves and other features? 






-U 


-2J 





For features other than eReserves? 



-U -2J ^ 



7 



What are you doing with Blackboard now? 



8 



What are the strengths of Blackboard? 
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Blackboard Survey - Spring 2006 



Basic Survey (Page 3 of 4) 



10 




I am interested in participating in further discussions about Blackboard and/or 
course management systems. You can email me at 




Would you like to provide us with details about your experiences? 
^ Yes, I want to continue with the survey. 

^ No 




Blackboard Survey - Spring 2006 
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Detailed Survey (Page 4 of 4) 

This section will take approximately 10 minutes to complete. 

13 

Indicate if you use any of the following types of media in Blackboard. Please 
check all that apply. 

PowerPoint 
^ Images 
^ Audio 

^ Streaming Audio Reserves 
^ Video 

Other - Describe: 



14 

Indicate the factors that influenced you to use Blackboard: 

To enhance my classroom teaching. 

To improve student learning. 

To help with a pedagogical need or challenge. 

To complement my interest in technology. 

To keep up with my peers. 

To meet the expectations of my students'. 
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^ To meet the expectations of my department chairperson. 



^ Other - Describe: 



15 



Has Blackboard been a positive enhancement to your face-to-face teaching? 



,yesj_NOj 

Why or why not? 




When you use Blackboard, has your students' engagement with course 
materials and/or classroom discussions changed? 

^ They are more engaged. 

^ They are less engaged. 

^ No significant difference. 

^ Explain: 



Do you think you are able to accommodate more diverse learning styles among 
your students since you started using Blackboard? 



_y£SJ_NOJ 

Why or why not? 
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In your experience, how effective is using Blackboard for... 



1 

Vei"y effective 



2 

Effective 



3 

Not effective 



4 

No opinion 



...organizing your course material? 

1 2 3 , j4 



...allowing students to submit their work to you? 

1 , 2 . 5 



...allowing students to check class progress (grade book)? 

1 , 



...providing electronic reserves? 

1, 2 3 4 



...preparing students for class? 

1 3 4 



...encouraging discussions outside of the classroom? 

1, 2_. 3 4 



...communicating with students (email, announcements, etc.)? 

1 2 . 3 . 4 



...helping students to collaborate on group projects or assignments? 

j4 



...challenging students to learn independently? 



19 
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Groups 

-U 




-U 


-£J 


Grade Book 








-U 




-U 




Wiki 








-U 




-3J 




Blog 








UJ 


-IJ 


-U 


-U 


Accessible from off 


-campus 






JJ 








Integration with eServices 






-U 


-2J 






Easy to learn and use 






-U 


-2J 


-3J 





Describe any other features that are important to you when considering using a 
tool such as Blackboard in your teaching. 
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Indicate what you think are the major advantages of using a tool such as 
Blackboard in your teaching. Please check all that apply. 
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^ Being able to organize and distribute classroom materials. 



^ Facilitating communication with students outside the classroom. 






Having course materials readily available to students at any time they 
want them. 






Having secure course materials, unless I choose otherwise (students 
must login to access your materials). 



Students being able to see their grades at any time. 



Other - Describe: 



Indicate what you think are the major disadvantages of using a tool such as 
Blackboard in your teaching. Please check all that apply. 

^ Does not integrate with eServices. 

Not enough flexibility - there are limitations in functionality. 

^ It takes a longer amount of time to develop my class or classes. 

The system lacks reliability. 

It gets in the way of my teaching practices. 

The students know more than I do about the technology. 

Other - Describe: 
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Indicate what would change your mind about using Blackboard. Please check 
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all that apply. 

Additional training for me. 

Additional training for students. 

Make it easier for me to use. 

Make sure it's available all the time. 

Give me time for developing my class or classes. 
Give me more features. 

Other - Describe: 



Indicate other online course tools like Blackboard that you have used. Please 
check all that apply. 





WebCT 




Discus 




On Course 




Wiki 




Sakai 




Moodle 




Blog 




Other - Describe: 
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Do you have any questions, comments or suggestions about Blackboard or 
25 course management systems? 




APPENDIX G 
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University #1 
Faculty 



Evaluation 
Cost 
Support 
Easy to use 
User-friendly system 
Pedagogical features 
Simplicity 



Selection 



Positive process 
Support vs. cost 
Clear process 



Adoption 



Easy features 



Caution 



Encouraged to use 
Fear of moving courses 



Diffusion 



Money to support 
Anxiety to use 
Reluctance amongst faculty 
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Staff 



Evaluation 



Ease of Use 



Resistance to learn 
Back end user 

Pedagogical Features 



Aligned with institution 
Adaptation to faculty 

Overwhelming 

Costs 



Security 



Selection 



Pedagogical Features 
Cost 

Ease of use 



Transitioning courses 



Adoption 



Slow transition 
Cost and funding issues 
Interest in features 

Pressure to adopt 



Converting courses 



Diffusion 



Organized process 
Faculty workshops 
Reassured training 
Quick process 
Attitudes changed 
New faculty required to train 




139 



Support issues 



Administration 



Evaluation 



Network compatibility 



Server compatibility 



Demanding process 
Meet needs 

Technical requirements 

Collaborative effort 



Intensive 



Selection 
Support questions 



Adoption 

Pedagogical features 
Incentives to adopt 



Diffusion 



Faculty Resistance 
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University #2 Case 



Support issues 



Faculty 

Evaluation 
User friendly 
Easy to use 
Pedagogieally useful 

Seleetion 

Cost 



Adoption 



Pedagogical issues 



Diffusion 



Technical issues 



Staff 



Eyaluation 



Costs 

Support faculty 
Marry people using technology 
Small group evaluation 
Initiative to jump on hoard 



Investigate new things 
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Selection 

Holistic selection process 
Systematic 



Features that will support learning 
Peer users 

Institutional goals 



Adoption 



Targeted faculty users 



Technical concerns 



Pedagogical use 

Technically frustrating 

Pedagogical goals 
Easy to use 
Training 
Support 

Resistance to change 

Diffusion 

Conducted Workshops 



No real goals 



Qyerwhelming 



Concerns with support 
Incentives to use 
Support in training 
Personnel and resource concerns 
Pedagogical feature problems 
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Administration 



Evaluation 



Broaden focus of technology 
Issues with use 

Support and reliability of product 
Focus on Pedagogy 



Selection 



Costs-non factor 



Adoption 

Challenge of Transitioning courses 
Support 

Influence from early adopters 



Diffusion 
Pilot tests 
Workshops 



Support from implementation and technology group 




